Organization OR Mapping The Sorted Details
In earlier articles, we talked through the steps involved in developing a great analytic question, investigating data, and the process required to further develop it. Now we enter the next phase — organization. It is a process that takes two distinct paths.
Sorting It All Out
The first path is basically sorting. It is a path that includes ordering and organization. Data requires structure to create real value. This can come in numerous forms. Segmentation, categorization, or simple aggregation are all viable techniques that group, cluster, and simplify the data.
More complicated processes may include scoring or decision trees. These are the (D) subprocess of the (O) step. Did you think we were done with recursion? Hardly. All the major steps come into play. Categories must be labeled and defined (I), good segmentation clearly follows a process (P), and great segments tell a clear story (S).
A short aside, while not as popular today as it once was — multidimensional cubes once ruled the analytic space. These tools often had languages and protocols that allowed users to place data “on rows” & “on columns”, but also“on pages” & “on chapters”. Organization creates a data story.
Mapping It All Out
The second path is visual. Is that a big distinction? Perhaps. Visualization (which is more of a storytelling aspect of analytics) is a channel. Visualized data is powerful but it also has different organizational constraints than general sorting. Mapping and sorting are related, but different.
Sorting is synonymous with grouping and aggregation, yet it is a type of analysis (breaking things down). Mapping is more predicated on order, division, and delineation, yet it falls more into synthesis (bringing things together). Confused? Don’t be. Just remember that –
at its core, analytics is mathematics
The easiest way to illustrate that is with a picture. If you think of math as purely equations, you will miss the point. If you focus on the organizational components of math, the analogy should come through.
Math is often reciprocal and confusing. Analytics is built on it, so you won’t avoid that feeling for long. More important — if you don’t feel the math component of analysis, it is a strong warning you are doing it wrong. I have a series of articles that challenge analysts to a mental crucible. Read this to go deeper in this space —
In the very first article, I mentioned that the very term analytics creates controversy in certain circles. I also jested they were the ignorant ones. I won’t hold back now. Because of that, I spent a full article defining “The Analytic Process”. Whether or not you agree, that definition set the bar for where this series was headed.
The analytic process, as I have endlessly repeated, is recursive. It also goes in forward and backward. Said differently — analytics includes the process of analysis (breaking things down) and the process of synthesis (putting them together). In my humble opinion, older colleagues who emphasized the former were segmenting the latter for executives and clients. Sadly — the latter may actually require greater education and discipline than the former. After all, computers and machines are getting really good at analysis (not so much synthesis).
It seems only fitting to interject this distinction now because —
A) it is a major organizational component of analytics
B) it gives me a framework to further develop my other segmentation — Sorting vs Mapping (which will get a dedicated article … coming soon).
SORTING uses techniques from synthesis to further analysis. We aggregate and group (synthesis) to better structure for deeper inquiry (analysis).
MAPPING uses techniques from analysis to further synthesis. We delineate, order, and divide (analysis) to better structure for development of models (synthesis).
Let me pause to insert a new visual. I am literally hopping over to paint.net to design it.
The Analytic Process is a strange loop. Thinking and reasoning often are…
While in no way an absolute, the organizational phase is often the pivot point in the process. It is the place where perspective is most often changed and leads to one those many recursions I keep droning on about.
It is also the connection phase. Each iteration through (O) should be seeding connections in the analysts mind. These were originally planted in the (I) & (P) phase but, as those tested and vetted in the (D) & (P) phases, new linkages and associations occur. And so (O) can be a nexus.
One of my favorite tools for teaching young analysts how to get comfortable with organization is the pivot table. Organizationally speaking — it is the equivalent of sliced bread. Although, I have always liked slicing my own bread, so to me it is actually much better. Pivot tables allow you to both slice the data and aggregate it, then pivot and start all over.
We have barely scratched the surface of the organizational phase of the analytic process, but we did organize it! Call this an introduction. Stay tuned for more articles to carry us deeper and farther. Remember that articles can only stage the major concepts. You have to get hands-on to really learn. Great mentors can sort and map this space for you. But hopefully, by now, you knew that. So thanks for reading! The next phase is coming soon…
Gurupriyan is a Software Engineer and a technology enthusiast, he’s been working on the field for the last 6 years. Currently focusing on mobile app development and IoT.