More companies are looking to move from relying on intuition to relying on data-driven or data-informed approaches. One of the biggest obstacles to this process is creating meaningful metrics based on the data. The data collection itself is usually easy enough, but making the data paint an accurate picture of what is happening can be a lot harder. The same data set can lead to multiple conclusions depending on how it is filtered and processed.
The choice to implement an analytics or metrics process can send shivers down the spines of workers. There is a trend for efficiency efforts to either try to create robots, or to come before a layoff as the ship goes down. The human element needs to be looked at and respected when creating metrics.
Bad metrics are more destructive than no metrics. Not all tasks are created equally and have to be viewed through different lenses. What do you do when people try to game the system? Focus on process and not targeting people, especially at the beginning. Make your metrics matter, make sure they stay relevant, and keep metrics from dehumanizing your employees.
Your Wish Is My Command
Most people want to be given instructions on what to do at their job. They don’t want to be left to wonder or to figure things out. They want clear instructions, a reason why they’re doing something, and a way to do it. If you give them stupid instructions, you’ll get stupid results. This is one of the first principles to consider when making metrics. People will do what you say whether it follows your intention or not.
This can come into play even more with malicious compliance. Ever been given a monumentally stupid task which you know will go bad and told to “just shut up and do it” after making your objections known? I know I have, and I know I took great delight in doing it to the fullest following each and every line item to the letter. It cost the business a client.
If you make stupid metrics, people will still follow them. They may even maliciously comply and follow them more than they would sane metrics. The difference is you’ll end up killing your business if you don’t adjust course. The instructions have to be clear and have to make sense or they’re going to backfire.
Gaming the System
One of my first experiences seeing metrics backfire horribly was in a help desk environment. Management decided to arbitrarily implement metrics without looking at the data fully. They decided that all they would measure was the gross number of ticket closes. They even went a step further and put people’s jobs on the line before looking at the numbers too closely.
What do you think happened? Technicians fought (almost physically at times) over password resets and all of the easy issues. Hard tickets ended up becoming a game of hot potato with techs fighting to not do them. People even poached tickets off of each other’s boards or fought to answer certain calls to steal closes. The help desk became complete chaos.
By treating each ticket as equal to each and every other issue, they punished their top tier technicians and rewarded people who played games. They set up rules which encouraged a specific result, and got exactly that result. There was a massive exodus of employees and clients due to the lack of forethought that went into the metrics. The wrong metrics ended up causing substantially more harm than no metrics ever did.
Not All Tasks Are Created Equally
This issue would not have happened had they considered whether the tickets were equal or not. By treating a password reset the same as a complicated Office reinstall on a machine with OS issues, they decentivized taking on the tickets which required any real work. Clients were angry because they didn’t get help in a reasonable amount of time. It should come as no surprise that not all tasks are created equally, and they have to be treated differently, but the difficulty comes in properly quantifying them.
How do you properly quantify a given task? For our help desk example, a password reset might take 5 minutes for the whole process in one environment and hours in another (AD has issues or similar). How do you differentiate these? There has to be a proper process which can reclassify events after an initial triage process. There also has to be a process to reclassify events at the end if they end up being outside the normal scope. That AD issue which made a password reset take hours is no longer a password reset or else your data is poisoned. By extension, this means your metrics are also poisoned.
The Pareto Principle shines here. Identify the 20% of issues which make up 80% of the work and assign the metrics to them first. Work out where these issues lie and use them as the baseline for comparing your other tasks. Not all tasks are created equally, so target the problem issues first.
Building Metrics
Metrics are typically determined by a mix of Key Performance Indicators (KPIs) and overall goals for an organization. KPIs should be broken down from individual goals and objectives on both a departmental and company basis. A sales person’s KPIs may be composed of the number of sales, number of retentions, etc. while a technician’s KPIs will probably involve ticket closes, engagement time, calls, customer satisfaction, etc. How do you come up with these goals, how do you quantify them, and how do you make them reasonable? Focus on the most common issues first.
What is the average for the organization at present and what is the average for the industry’s target niche? Getting the current average for the organization for each given goal should be the first step. Start with the core issues which take up the majority of time and work out. It can be near impossible to get numbers for the industry in general, but certain numbers can be extrapolated from certain public data. It depends on the industry and the level you’re at. There are also information services which can provide some of this information.
Build metrics off of what you can know to start out with. What does a good employee look like at your company? What do they do in a given day? Is there something that makes them efficient or inefficient in their process? What tasks use the most time? Build a profile of what each individual should be doing for a streamlined process, but make sure it is something attainable. It’s easy to say the salesperson needs to close 20 sales a month, but it’s not going to happen in December for most industries.
When building metrics, start with the core competencies of a given position. What should this position be focused on? Once you figure out what they’re supposed to be doing, quantify it, and then you can look at what else they end up doing. Are they doing admin tasks? How do those affect their primary job? Add those in but make sure that the time adds up to something sane. Make sure that the employees know what their metrics and KPIs are, and why they exist.
Shaping Metrics
As you build metrics for a given position, you’ll find that certain things in the implementation or process just don’t add up. The metrics need to be shaped to reflect what the current position entails rather than what you want at this phase. Find a way to measure what you have, and find what you want long term. You can want to build a mansion, but you won’t get far if all you have on hand is mud.
How can you shape the process to change the metrics you have into what you want? Some places are fond of just reorganizing everything arbitrarily, but too many changes will scare off employees and usually ends up as a fiasco. People need to be trained as they go and have a way to see if what they’re doing is what they’re supposed to do. Too many changes and you risk rocking the boat too hard. Make gradual changes to the process and see how it affects the process. If a metric change hurts the process, fix it.
By making gradual changes, you can account for a single variable at a time. If you change 5 things, how do you know if all 5 are necessary, or if 4 are just coincidental? Even worse, how do you know that 1 isn’t just super efficient and the rest actually hamper its efficiency? As the process is ironed out, you can make larger changes, but in the beginning, slow down and focus on process.
Auditing Metrics
Metrics need to be shaped and adapted to the environment just as the environment is adapted to the metrics. Process needs to grow around the metrics as much as the metrics need to grow around the process. You can’t expect a call center to answer as many calls if they’re not ever allowed to tell the customer goodbye and have to wait for them to hang up. If you want a change in the metrics, you need to change the process, and vice versa.
Audit your metrics and make sure they’re doing what they’re supposed to be. Metrics can be great on launch but quickly become outdated as things change, or even worse, just gamed relentlessly without an audit. Spot check data to make sure that things are going the way they should be. Is the data being filed correctly? Is a password reset being set as a password reset or some other support type?
Auditing metrics allows for process consistency and for a process to be more adaptable. Issues can be headed off before they spiral out of control. Make sure there is training about what needs to be done and why. Also, make sure a process exists to manually override KPIs and similar on audit. If a client is angry something isn’t possible, should that reflect poorly on the person who got left holding the bag? If it does, it shows your employees they’re nothing more than data and they’ll avoid tasks which might hurt them even if they follow process.
The Human Side of Metrics
Metric and analytic processes can garner animosity even when implemented properly. Metrics and KPIs are typically associated with a dehumanization process which reduces employees to a number. Sadly, this isn’t without reason. The ultimate goal of setting up metrics and being data-driven or data-informed is to remove the human element from the data. The problem is how the data is applied back to the human element.
Metrics are about improving the efficiency of an organization and improving the efficiency of process while intentionally ignoring how a human acts. The metrics should be designed with human fallibility and human elements built. You can’t expect 8 straight hours of work in a day, especially not in a mentally or physically challenging job. It just isn’t feasible and the metrics have to be designed with this in mind.
When you dehumanize the process, the goal is to remove the human element from the data, not to turn the human into a robot. Humans have human limits, and no amount of number crunching is going to change that. Use the data and the metrics to help motivate people, not to just punish them for being a human.
Acting On Metrics
Once you have your data and build profiles of the various parts and pieces of your company, you need to have a way to act on what you find. Metrics should first be used to adjust process. As you make the process more efficient for both you and your employees, you can begin to focus on the individuals. Are they doing what they’re supposed to and is something not working out? This might be a time for more training or reassignment.
You can change what a person does, but you aren’t going to change their personality. Some personalities just don’t fit certain positions. Keep in mind that some people want to comply and others want to make decisions. People will adapt if they need the job, but then you take them out of their element. Is it worth it?
Metrics get such a bad name because many companies default to hiring, firing, promoting, demoting off of metrics out the gate instead of focusing on process first. The process needs work and the metrics need to be solidified before you start even considering targeting individuals. If someone has never been shown what or why they’re supposed to do something, whose fault is it? Square away the process from what you find, then reassess what people are doing. Start from the institution and work in as people have a chance to adjust to the metrics and new process.
Conclusion
By focusing on a top down approach, you show people that the metrics aren’t there to dehumanize them and reduce them to numbers. You also show that the goal is efficiency of the company rather than giving the impression of targeting individuals. You don’t go and buy a house without an inspection first; the process of building analytics and metrics around your process give you insight into what works and what doesn’t and where you are, but it doesn’t fix the holes in the roof. Square the problems away before you go down the wrong rabbit hole.
To act on metrics, you need metrics worth acting on. If you give the wrong instructions, people will follow them. Measure your tasks and don’t compare apples and oranges. Make sure people can’t game the metrics. Shape the metrics to reflect the reality of the process, and audit the metrics, both to shape them and make sure that the process works as expected. Most importantly though, make sure your metrics account for human nature. Make metrics which actually matter.