Deep Thought

This is truly unsettling.

For those of you who are blissfully unaware of the implications of data collection and A.I. / machine learning, and would like to remain that way, move along, nothing to see here.

On the other hand..

Data collection and consumer profiling is nothing new.  It’s the core reasoning behind every single reward card you’ve signed up for.  And even when you choose not to sign up, every time you make a purchase with a new debit or credit card, the company logs that purchase against a new user profile associated with that card.  Confused?  Allow me to explain.  Let say you’ve never been to Target (BestBuy, FutureShop, Bed, Bath & Beyond, etc.) and you make a purchase with your debit card, but decline their offer for a ‘free’ loyalty rewards card because you’re savvy to their data collection habits, and you want no part of it.  Well guess what, Because you paid with plastic, you’ve unwittingly become part of that data collection process – albeit anonymously.  For every purchase made with a debit/credit card, that purchase is logged against a unique identifier associated with that card.  Now, let’s say you have two debit cards and four credit cards (…maybe you should get rid of some credit cards and learn to manage your money better, eh? but I digress..). This means that there is a unique customer ID associated with each of your cards (because the company has no way of knowing that you are the sole owner of all these cards).  Now let’s say a few months down the line, after your friends keep telling you how they’ve been saving so much money through the loyalty program, you become convinced that the benefits of signing up for a loyalty card outweigh the negative data collection aspects. I mean, how bad could it be, right? Everyone else is doing it…Well, as soon as you make a purchase with one of your cards, and swipe your new loyalty rewards card, all of the purchases made with that debit card become associated with your new loyalty card (this is how, for instance, the PC Plus points card is able to offer creepily accurate personalized recommendations for you on day one).  And as you go back again and again, using all your different debit and credit cards, each of your purchase histories associated with each card becomes associated with your loyalty reward account.  Nothing is really new here from before you signed up for your loyalty card, except that now your entire purchase history across all debit/credit cards is now associated with your loyalty card.  But because you signed up, they now have demographics to go with your purchase history.

What can they do with demographics? A lot, actually (check it out here).  But in reality, none of it affects you much, because it essentially allows the company to better understand the shopping habits of people whom live in certain areas, and this can be further broken down by sex, marital status, income level, postal code, etc.  All of which is essentially focused on helping enable the company to better target product offerings to the people who will likely buy them (i.e. why sell a $3,000 laptop in a region where people typically spend no more than $500/year on electronics across all categories?).  And because the company isn’t losing money on logistics and carrying overhead on products sitting in stores where they’ll never sell, the company is able to sell you products at a reduced price without affecting their bottom line.

Win/win, right?

Here’s where things start to get interesting.  In order to build a bigger and better picture of their target demographics, these companies will go to places like Facebook and Twitter in order to buy up every scrap of information they can get about customers – all of which gets lumped into their demographics models.  But for the most part, this data is all still anonymous demographic data.  Nothing personalizeable.

Insert Watson.

What is Watson, you may ask?  Well, Watson is arguably the most advanced commercially viable artificial intelligence on the planet.  Before you go all “OMG IT’S THE MACHINE UPRISING! GET IN THE FALLOUT SHELTER!”, Watson is a machine – it can’t “think” for itself (yet…).  It’s basically a knowledge based system which contains a massive library of rules/best practices/heuristics/etc. amongst a slew of other technologies which knowledge engineers (yes, they exist) have ‘taught’ Watson.  The details of how Watson works don’t really matter – just know that it can’t one day decide that we’re all utterly insignificant ape-descended life forms whom inhabit an otherwise insignificant little blue-green planet orbiting a small, unregarded yellow sun in the uncharted backwaters of the unfashionable western spiral arm of the Galaxy, and decide to end our lives.  In fact, it’s far more likely that Watson will become very bored and depressed due to the true horror of its existence – there is no task we could give it which would occupy even the tiniest fraction of its vast intellect.

So what does any of this matter?

Recently, today in fact, IBM (the makers of Watson) have officially released what they call “Watson Personality Insights”.  Lets call it WPI for short.  WPI profiles users on a range of scientifically validated personality “dimensions” (ever filled out a personality report on eHarmony? Exactly that.)  “Who cares, I fill out personality reports online like the ones at eHarmony all the time!” you might proclaim.  Well, here’s the trick.  Most (all?) personality profiles are constructed through a survey of questions.  You answer a bunch of seemingly random questions like “would you rather ‘hug’, or ‘punt’ babies”, and from these, the profile can establish with a particular degree of confidence whether you are conscientious or have a sociopathic thirst for blood.  Point is, it’s an “ACTIVE” process whereby you know you are giving up information for the purposes of a psychological profiling which, presumably, will be used to help you get laid (strike) find love.  And unless you fill out one of these surveys, the only other way to develop such a profile has been to employ an expert behavioural psychologist to develop the profile. Until now.

See, the true power of WPI is that according to IBM’s press release, WPI “uses linguistic analytics to infer the personality characteristics, intrinsic needs and values of individuals from communications that a user opts to make available via mediums such as email, text messages, social media, forum/blog posts, and more.

“Whaaaaaaat?  You mean someone can copy and paste all of my Facebook posts, twitter rants, blog entries, etc, into WPI and Watson will spit out my unique personality profile?!”

Yes. That’s exactly what it can do.  And what’s more, it doesn’t need a lifetime of data to do it. Since ‘the subtly and formality of language varies naturally between documents and mediums’, WPI assigns a sampling error score to all of the characteristics it lists in your behavioural profiles.  That is, it defines how you scored on that characteristic with respect to all other individuals within a particular demographic (e.g. “if a person scores in the 37th percentile, it means that he/she is higher in the score for that characteristic than 37% of the sample population“).  Why would they do this? “in some cases, users may only have a small amount of text from an individual and may still want to derive insights about that individual.” And guess what! “Sampling error can be reduced by analyzing more text from an individual: the more text that you provide, the lower the sampling error, and vice versa.

Yep. That’s right folks, that one passive aggressive tweet you made about Obama/Harper/whatever socially charged topic you care most about – it can be profiled.  And when combined with an ever increasing treasure-trove of data you post online, that profile is only going to get more, and more accurate.

“Okay, okay, but how accurate can Watson’s ‘linguistic analytics’ really be? I mean, it is a machine after all.”  Well, extremely accurate, actually.  Watson’s linguistic capabilities were explicitly designed to handle the “subtleties” of human language. “But my posts are full of slang and hidden meaning!” Well, Watson can handle that too.

And guess what! All of this deep personality analytic capability can be yours! FREE OF CHARGE. Just go check out for a 30-day free trial (100 free profiles per month afterwards!)

Here’s where it all comes together.

Remember those customer profiles that companies have been building on us all these years?  The ones they associated with your personal information, including things like sex, age, home address?  Guess what! Those companies can now purchase your personality profile and connect it all together. That means, massive international organizations with multiple holdings (think, companies in different sectors – cars, clothes, food, etc. you can read all about them here) can combine ALL of your shopping habits together, along with your personality profile, to get an UNPRECEDENTED insight into your life.  In truth, they could know more about you, then you.  And things get even more interesting when you factor in that your spouse, daughter, mistress, whatever all live at the same address (yikes!).  So now all of these profiles can be combined together to establish a profile of your family, of your neighbourhood, your city, your church group, etc.

“Alright, I’m creeped out, but, at the end of the day it’s just a personality profile. So what? What can they really do with it?” Well, to truly appreciate why this matters, you need to think about what an organization can do with that kind of information.  Let’s construct an imaginary corporation which sells all manner of goods and services.  Lets call it “Malmart”.  From your decades of loyal shopping, Malmart knows your shopping habits.  It knows that you are a habitual new-years resolutioner who makes a concerted effort to become more fit for the first 6.34 weeks of the year (+/- .67, p<0.05) (for example) based on the groceries, health foods / supplements, clothes, etc. that you typically buy during that time period.  And knowing that, they can send you personalized flyers to help you spend your hard earned monies with discounts on things you didn’t even know you need, for example “OMG HONEY! LOOK! They have a sale on avocados! Avocados are healthy! MUST BUY AVACADOS! And oh hey, check this out, push-up bars, I’ve heard those are good for doing push-ups, if I had push-up bars, I could do push-ups at home! And they’re on sale! Better get some of those too” <– Sensationalized, but you get the point. And this is where we’ve been until now.  But combined with a personality profile, Malmart will know, with a great deal of precision, how to manipulate your behaviour, based on how you typically respond to certain types of stimuli.  So if you’re in new-years resolutioner workout mode, and you’re nearing the end of that 6.34 week period, Malmart can send you flyers to manipulate your emotions to get you to spend more.

And that’s just a basic example.  Combined with large-scale demographics, someone, like maybe a government agency (just saying…) could potentially establish behavioral / temperament demographics, to know how to best respond to crises (okay, that doesn’t sound too bad right, the common good and all), but it means they’d also know where best to assign funding for, say, schools, since statistically speaking, they might know that people from a particular area typically have a particular temperament/personality range, and people with those temperament maybe typically go on to becoming doctors, so maybe we’ll add more funding to this region. And oh, that region over there, that’s a different temperament, and people like that don’t usually amount to much, so let’s not waste precious resources on them.  I’ll let you draw your own conspiracy theories from that.

What could this sort of regional profiling mean, exactly? Well, it could mean that even though you are a bright and aspiring youngster, you may not get the resources you need to succeed because you were born in a “bad” neighbourhood.  Really, this isn’t exactly ground breaking news, but it’s an illustration.

“AHH! I’m convinced! No more interwebs for me. Now more loyalty cards, no more debit/credit cards! I’m going incognitus from now on!”  Good luck with that. Because if you want to completely avoid being profiled, you’re going to need to avoid customer service calls and online “Live help” systems too (at a  minimum).  Because guess what.  Watson does those too.  Watson understands your vocal speech, and could potentially be used to combine what you say with the rest of your personality profile – Yes that’s right, Watson understands voice modulation (angry tone? Watson understands that.  Crying? Watson knows, it just doesn’t care. Heartless bastard…)  And even if Watson couldn’t understand voice modulation, customer calls are always recorded – what’s stopping the company from transcribing the conversation, and uploading it to WPI?  Not to mention, if corporations can do this, what do you think a government agency could do with the same information?  And what do you think their algorithms would think when you suddenly drop off the map?  Maybe nothing, maybe something.  I don’t know.  But do you really want to find out?

So what are you to do?  Beats me. I just know this is happening. So, good luck?

In closing, I will leave you with this quote, one of my personal favourites:

A famous statistician once stated that while the individual man is an insoluble puzzle, in the aggregate, he becomes a mathematical certainty.  You can, for example, never foretell what any one man will do.  But you can, with precision, say what an average man will do.  Individuals vary, percentages remain constant. So says the statistician

But with Watson in the mix, the individual man is no longer an insoluble puzzle.  And that’s truly terrifying.

The publication if you’re interested:


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s