Just curious, if you could highlight some of the problems you've experienced with each "generation" of looker reporting? Was there any specific issues that encouraged the re-implementation?
1. Complex joins and poor performance.
2. Not pre-computing important attributes made them hard to slice and dice on (because they weren’t true dimensions)
2nd gen:
1. Really unmaintainable. Like....really unmaintainable. A rat’s next of ad-hoc common table expressions, that referred to each other in undocumented ways, used inconsistent CASE statements to transform enums to human-readable values, inconsistent time stamp formats (and time zones), etc, etc, etc. Fields in different explores that were named the same but didn’t quite match and no one could explain why.
2. All of it was SQL in LookML and the persistence strategy (e.g. when were the CTEs running, were they writing to disk or ephemeral, etc) was not manageable.
If we had been smarter, we would have introduced DBT after gen 1 and not gotten into the gen 2 mess.
There’s a ton of other nuance here, but that’s the high level.
> If we had been smarter, we would have introduced DBT after gen 1 and not gotten into the gen 2 mess.
Is this a roundabout way of saying you wouldn't have exposed looker to the underlying staged data?
We have two layers within our data warehouse: The ingestion/staging layer, and the organized/cleansed/segmented layer. We only give Looker projects access to the latter.
We still have a lot of chaos within our Looker projects (views with select star, dimensions for every column bc of select star, hard-coded schemas, etc...). Slowly working it out though.
