Metadata
- Author: Torsten Walbaum
- Full Title:: The Ultimate Guide to Making Sense of Data
- Category:: 🗞️Articles
- Document Tags:: Metric trees,
- URL:: https://towardsdatascience.com/the-ultimate-guide-to-making-sense-of-data-aaa121db1119
- Finished date:: 2024-06-15
Highlights
Quantifying the lag It’s worth looking at historical conversion windows to understand what degree of lag you are dealing with. That way, you’ll be better able to work backwards (if you see revenue fluctuations, you’ll know how far back to go to look for the cause) as well as project forward (you’ll know how long it will take until you see the impact of new initiatives). In my experience, developing rules of thumb (does it on average take a day or a month for a new user to become active) will get you 80% — 90% of the value, so there is no need to over-engineer this. (View Highlight)
The main problem with monthly metrics (or even longer time periods) is that you have few data points to work with and you have to wait a long time until you get an updated view of performance. One compromise is to plot metrics on a rolling average basis: This way, you will pick up on the latest trends but are removing a lot of the noise by smoothing the data. (View Highlight)
Looking at the monthly numbers on the left hand side we might conclude that we’re in a solid spot to hit the April target; looking at the 30-day rolling average, however, we notice that revenue generation fell off a cliff (and we should dig into this ASAP). (View Highlight)
- Accounting for seasonality (View Highlight)
When you see a drastic movement in a metric, first go up the driver tree before going down. This way, you can see if the number actually moves the needle on what you and the team ultimately care about; if it doesn’t, finding the root cause is less urgent. (View Highlight)
When dealing with changes to ratio metrics (impressions per active user, trips per rideshare driver etc.), first check if it’s the numerator or denominator that moved. (View Highlight)
By segmenting across the following dimensions, you should be able to catch > 90% of issues: • Geography (region / country / city) • Time (time of month, day of week, etc.) • Product (different SKUs or product surfaces (e.g. Instagram Feed vs. Reels)) • User or customer demographics (age, gender, etc.) • Individual entity / actor (e.g. sales rep, merchant, user) (View Highlight)
One of the most common sources of confusion in diagnosing performance comes from mix shifts and Simpson’s Paradox. (View Highlight)