More accurately, to legacy enterprise management.
Let’s say the following directive comes down from on-high: “Hey, our CEO wants us to provide better financial metrics reports and a dashboard that management can see to show real-time stats about the company.”
I mean… Sure! Yay, digital transformation, modernization, mobile friendly, all that good stuff!!
So, I have some thoughts on this, because I’ve seen the current state of things in small-medium enterprise, and am anxious to help improve that state to provide better value to the business. To
misquote Dennis Miller, I don’t mean to on a rant here, but…
First topic: Reality Check
It starts at the top, with a couple realizations:
- Data is ever-growing.
- We need to get smarter about managing its growth, including archiving/retention schemes, data warehousing, etc.
- This involves compliance regulations and operational resources.
- We need to ensure compliance with biz standards and data shelf-life.
- We need to automate as much as possible to avoid over-burdening our human resources (and to some extent our servers too).
For example, you can’t expect the same response-time for a query into 10-year-old financial data as you do for 1-year-old data.
- Traditional SSRS (SQL Server Reporting Services) is an operational time-sink.
- We spend way too much time assigning access, creating redundant “on demand” reports, and making seldom-used email subscriptions.
- We’re probably running on an old version, say 2008R2
- Vast improvements have come to the MS Data/BI platforms in the last decade and we need to take advantage of them.
- It’s not mobile-friendly at all; it’s not even modern-browser friendly, as some of its UX elements are still explicitly functional in Internet Explorer
- We tacked-on some 3rd-party application to attempt to bring some data-warehouse functionality into the environment, but only 1 person “knows it intimately” and is comfortable developing new reports with it.
- Our ERP system, in its current state/version, is a tangled mess, to the eyes of a DBA & query-writer/report-writer.
- We’ve bolted-on so much customization and special-configuration that it’s not suitable for stock/canned reports from the vendor, even if we upgraded to a version of the app that had a decent reporting engine.
- We can’t even decide on very basic things like “What is a ‘unit‘ of production?”, or “What are the different areas/groupings we break-out for revenue metrics?”
Ok sure, maybe we can agree on what those groupings are, but we can’t even get a consensus on what we call them!
Second topic: Single Source of Truth
We need to agree on a standard, documented, official set of business rules that answer such questions as “how do we measure revenue?”, “what are our different sub-orgs/departments/groupings for how we report on revenue?”, “what is ‘production output’ and how do we measure it?”, “how do we calculate bonuses for this group of employees?”, etc. More than that, we need to agree on naming things – we need a common, consistent nomenclature and understanding of what it means when someone says “N# Units”, “Department X” or “Order Aging” or “Membership Level” or “Bonus Type Y”.
And even more than that, we need to map those concepts to concrete, documented rule-sets that are manifested in the data somehow (from the simplest example, a “look-up table” or “reference table”, to the complex examples like a “data mart” or “analysis cube” or “ETL process”). This concept is sometimes called a “data dictionary”, which kinda belies its complexity, because it’s really more of a “data encyclopedia” – it needs to document what, how, why, & when.
What our concepts/terms/data-points mean, how they’re used, why they’re useful, & when they should be used.
Third topic: KISS and KPI’s
Management reports need to be simple. Yes, there are power-users who want the detail, and there are auditors who in fact require the detail. But your average C-level (or even P/VP-level) exec doesn’t care about that stuff – they want very simple answers to deceivingly simple (i.e. can be very complex under-the-hood) questions, like “How much money did we make this quarter for department X?”, or “What kind of productivity bonus do I give to group Y?”. But that’s just the beginning – that’s descriptive analytics. What they really want, but are sometimes too afraid to ask, are more powerful questions, like “How much money can I expect to make in market Z or state XX?”, “What are our expected new loyalty program memberships, and how much will they profit us?” — predictive analytics. (And we’re not even going to touch prescriptive analytics yet, because you’re not ready for that.)
KISS means we need to try our best to hide the nitty-gritty details and “under the hood” logic/calculations from the end-user or report audience. But, that means fully knowing and understanding those details and rules and logic flows so that we can implement them!
KPI is Key Performance Metric. That’s the golden nugget, the one piece of information that the manager/report-viewer ultimately is after, the thing that makes them go “Got it! That’s the answer I was looking for!”, so they can make their business-decision and move on with their day. These aren’t necessarily just single numbers (like an overall revenue figure); they can be pie-charts, bar-graphs, a clear & concise grid, or whatever makes the most sense for the business-problem/business-decision at hand.
This all sounds fantastic, right? So what’s the catch?
Fourth topic: Time & Effort
Time is money, which is resources, which is people, learning, training, developing, implementing, testing, validating… rinse, repeat. You don’t put that all on the shoulders of a lone DBA; that life-cycle touches many different disciplines and team members – managers, business users, accounting folks, marketing people, analysts, developers, testers, operational leads, and yes, of course, all of IT infrastructure (helpdesk, engineering, DBA). And you don’t just buy a box off the shelf at your local software retailer and say “look, we’re gonna implement Tableu!”, wave a magical IT wand, and call it day.
Now we, as technologists, are more than willing to learn and educate ourselves, but…
There needs to be a matching dedication from the business to that effort, and to the platform(s) that is/are chosen.
That means, in concrete terms, a few things:
- Training budget & resources
- Conferences, courses thru online training providers, cross-team collaboration.
- Product & technology investment
- Upgrades, net-new products, whatever is needed.
- Time allowances & agreements
- Dedicated scheduling where the “daily grind” operations take a back-seat and we can focus on the new stuff.
- Support from SME’s
- The ability to call-out to a qualified expert when critical questions or roadblocks arise.
- Can be contractors, consultants, service-providers, or platform-providers. The point is, you only use them if you need them, so you keep the cost relatively low.
That’s if you’re dedicated to in-house team/ability build-up. If you want to outsource, you have a different set of challenges:
- Contractors are expensive!
- Their requirements are exceedingly rigid.
- They’re likely to scoff (yes, even outright laugh) at the quagmire of data & logic & rules that we’ve created and/or want to build into our “magical reporting stack”.
- They’ll still require that same product/tech investment.
- No contractor is going to accept your old legacy SSRS instance as a baseline for building a modern, responsive, effect reporting system. The first thing they’ll say is “upgrade that, & come back to us.”
- Likewise for your legacy ERP system – sure, it’s a little less obsolete, and there are probably plenty of shops running it & developing on it, but good luck getting new-hire contractors to embrace it; at best, they’ll begrudge it; at worst, they’ll charge exorbitant fees for having to work on such an old platform.
- Technical debt is their worst enemy.
- Like it or not, like most decades-old enterprises, we have technical debt up the wazoo.
- Contractors won’t work in a debt-heavy environment; they’ll insist you “fix the debt” and come back to them in a few months/years when it’s all happy & pretty & green.
Technical debt is our enemy, too, but at least we “own” it – i.e. we’re aware of it and we have ideas on how to fix it, if/when we ever get the time.
It’s like our city roads: at least we know where the potholes are, and how to avoid them.
My point, from this rambling and probably way too lengthy post, is this: We need to talk about this stuff. Yes, Mr. Manager, I know you already said that. Let me embellish:
We need to talk about this stuff, candidly, openly, broadly, deeply, cross-functionally (made-up phrase #2), even company-wide.
Because, while the end-goal is deceptively simple (“We want report dashboards!”), the underlying systems are complex, with lots of moving parts, requiring lots of knowledge (both domain/biz and tech), and lots of management (compliance, governance, automation, visibility/monitoring).
It’s not just a technology challenge. It’s a people challenge. It’s a cultural challenge. It’s an organizational challenge.
It’s a challenge that, when faced, met, and overcome, can lead to spectacular growth and success for all involved!
(And that’s my attempt to end this rant on a positive note. Enjoy!)
PS: No, I’m not happy about WordPress’s inability to understand the ‘style’ attribute of a simple <ol> tag, but I tried… so apologies if the outlines are not intuitive because each level is just another set of numbers, instead of Word-style outlining like 1.. a.. i.. etc. Grr arg!