Personalization of Content and the Power of Metrics
“Always in motion is the future.”
-(YODA, Star Wars Episode V: The Empire Strikes Back)
Predicting the future
Traditionally, predicting the future was a craft that belonged in the realms of magic and religion. Whether you were reading bones or stars, stuck inside a whale or speaking in tongues – seers and prophets had great power. People live in apprehension of what “may be” and who wouldn’t want to know the turn of a business deal, the time of their death, or how beautiful their bride would be?
Over time, science crept in and gradually replaced magic. As the credibility of science grew stronger, belief in the supernatural waned and skepticism replaced blind faith in the power of prediction.
But despite what many people believe today, predicting the future – isn’t always about magic. Just like science allows us to regrow limbs, fly the to the heavens and read minds, today, with the ability to calculate probability, we once again have the skill to predict the future, this time with science on our side. We’ve come full circle. Predicting the future through probability – or predictive analytics – is the ability to collect enough data in order to be able to identify behavior patterns and predict future similar patterns.
With a lot of experience, good soothsayers could look at people and “read their minds”, the more people they met – the better they became at it. In just the same way the more data you have, the more reliable your prediction will be. Mega-corporations like Google and Facebook, which have access to incalculable amounts of user data, have long since taken the lead in predicting user preferences and targeting them with additional content in order to monetize this data. In this article, we’ll take a look at some of the more accessible ways training teams can also leverage user behavior analytics for delivering accurate content that will help improve training and onboarding results.
I agree – predictive analytics lack the awe and splendor found in hoodoo ceremonies, but it sure make up for it with accuracy.
Personalization is taking over digital interactions
The most common and effective use of user data, industry-wide, is to use it to create personalized content. I’m referring to content in the wideset sense of the word: social networks, advertising platforms, micro-learning platforms, e-commerce platforms, and so much more.
Social networks are getting better at delivering organic/paid content based on what we previously engaged with. If I often “Liked” or commented on, Walmart’s Facebook page, I’m more likely to respond well to their promoted ads. Those patterns will eventually dictate the content that comes up in our Facebook, Twitter, and Instagram feeds and allow Facebook to target me with correlating content. Now let’s put predictive analytics into an even more advanced mode: Let’s say that 73% of Walmart’s Facebook page visitors also like to buy at Macy’s, Macy could use Facebook’s algorithms to automatically target every new Walmart page viewers, predicting a 73% chance that this would be beneficial for them!
The distinction between paid and organic content on social platforms is becoming more and more blurred. What makes that possible is the razor sharp targeting based on personalized preferences. When we, as users, are engaged with what we read, we’re more inclined to become immersed in it and even more enthusiastic about paying to stay at that level of engagement and interest.
The same goes for micro-learning. By its nature, micro-learning is designed to be bite-sized assistive content. This medium’s purpose is to blend seamlessly into existing products and make learning seem like an organic part of the product experience. In micro-learning, words are chosen carefully because any wrong turn may result in the user losing patience and wandering off the page.
Delivering personalized content is what can turn a “meh” micro-learning experience into a winning training session that wins the user over. These small bites, viewed by and engaged by countless trainees create a huge amount of data. Countless bits of data can be cross referenced with other training data and with additional user information, and using analytics, allow us to offer predictive understanding about future training and support needs.
Getting it juuuuust right: personalized training
So how do we go about delivering the perfectly tailored content that would guarantee users’ engagement? How do we apply that to training environments?
While we take an eagle-eye view of the whole body of information to really understand users’ preferences we must dive into the huge database that is user actions. Each user activity is aggregated and added to the equation, we need to understand which actions was repeated, where the user got stuck and which actions she completed. Once this data is collected we add to our “magic” cauldron as much user information as we can get our hands on: role, position, history of usage in the product. Even level of technical savviness.
- While each user is unique, each user is also part of the whole and the whole is built of countless users.
- While each user is unique, each user can also be classified via “types”, “sub-types”, “sub-sub-types” and so on…
- Predictive analysis is based on understanding what most users of certain types prefer
- The more data we have (the more people and actions) – the better we get at predicting activities and needs
- More and more powerful software and hardware can contain and manage huge amounts of data, sift through them quickly and make complex computations to glean meaning from it all.
- As these system gets more and more complex – reviewing huge amounts of information from billions of users with hundreds of billions of activities – their ability to second guess your next need/action come close to perfection.
- With all that data at our fingertips, we can easily predict which assistive content would most appeal to the user, what would be just-in-time help and what she would find most engaging. Once we deliver that content, we can measure the level of engagement, repeat or correct, according to the success stats.
Training evaluation done right
While training evaluation is a school of analytics that has been around for a while, coming by proven successful methods is harder than it seems. In this sphere, the Kirkpatrick evaluation method is considered one of the founding fathers of modern training evaluation. The idea behind the Kirkpatrick method is simple: he differentiates between 4 levels during the evaluation process: Reaction, Learning, Behavior and Results. This is a holistic approach that considers all aspects of user reaction to training.
One of the reasons the Kirkpatrick is considered an authority on training evaluation is because he insists on measuring training results and not just training metrics. Many training environments are conditioned to measure only training success, with a variety of training metrics: completion of training programs, tests designed specifically to examine if trainees internalized the material and similar metrics.
Measuring results means measuring the user’s performance during work routine after the training is concluded. Returning to personalized content – if we were to know how well a user applies her training to everyday tasks, we would be able to offer her ongoing micro-training that fits her level.
A few examples for post-training metrics:
Events have become the most popular and thorough way to evaluate user proficiency in SaaS platforms. Most analytics tools use it as a base parameter. Events are normally a series of actions that culminate in one specific action that is easy to track: creating items and saving them, generating and exporting reports, visiting certain areas in the platform.
How often does the user login? When was the last time they logged in?
How long do active sessions last? Frustrated users won’t last that long, opting out when the frustration from failure to get what they need from the software becomes too much.
Identifying usage patterns
For instance –
(~) Novice users spend a lot of time in the app, but they are still insecure and trying to master features, and so they spend much time in the same area (metrical indication: longer sessions on fewer pages.
(~) Advanced users spend more time in the analytics section, and are more confident about moving around in the platform to get things done (metrical indication: shorter sessions, multiple pages)
Users normally invite their team or colleagues to collaborate on a tool they feel confident with, when they are in a position to answer questions and tutor their peers”
You can read about it more extensively here: How to Evaluate Training Effectiveness (with the Right Metrics)
Best practices for training evaluation
Deciding what to do with the data can be quite challenging, but getting your hands on it is just as much trouble. There are plenty of tools out there to help you gather information on user behavior – the trick is to know which ones to combine for best results.
To get the right data, we first need to understand what we’re looking for. Identifying user behavior on web-based products means tracking user actions by pre-set goals and events. For instance: clicking “submit” in certain forms, completing certain fields, creating certain elements. Goals need to be things an analytics tool can track: buttons, fields, time spent on a page, elements viewed.
For getting a visualization of user behavior in your platform, the best option is session recording, like what Hotjar does. You can record and then play and inspect user sessions in your product. This is a terrific way to identify usability issues, figure out which your Calls to Actions (CTA’s) are less effective and in which flows users are getting stuck. Hotjar specifically also offer heat-maps, reflecting which areas are clicked more.
Now that we have the data, what can it tell us?
Tracking user interaction with those elements can tell you a lot about 2 things – product limitations and the user’s level of engagement with the product:
Product limitations can be identified only by accumulating enough user data to indicate a pattern. If a critical mass of users are having a hard time completing a certain task, that may mean it’s time to revisit the user experience design. Users don’t always recognize the root of issues they are having, and less tech savvy users may default to assuming the problem is with their skill rather than the product.
User engagement, on the other hand, is analyzed individually by user. Low user engagement can be the result of the user’s uncertainty as to the technology or lack of proper training. Optimally, your analytics platform would be able to do both: aggregate large amounts of user data and break down individual user activity history. Here’s an example, from Iridize’s own analytics platform, of 2 separate data sets that complete each other to show the full circle of user behavior data:
In that regard, user analytics can help change and even redefine supportive content, in that it offers insights into how users are interacting with your product from multiple angles, highlighting the perspective you need.
Many solutions today allow you to follow user behavior, parse their activities and analyze them, but real evaluation of training needs to have the ability to continuously see how trainees acted after training, over a span of time, how often they needed additional support and how quickly they mastered the various tasks.
Predicting user behavior only has meaning if we can deliver the content that improves the user’s experience – in product use, training and productivity. Today, we have the tools and capabilities to personalize that content in a way that creates a truly seamless user experience.
- We’ve collected user behavior, parsed it and combined it with additional information to makes sense of it all.
- We allowed our platforms to review this information and developed algorithms that, based on rules we generate, will provide users with additional content.
- Now we are ready for the next big leap – letting the system learn user behavior from the analytics and understand that behavior in order to generate new content – and that’s where Artificial intelligence steps in.