The ‘Modern Data Stack’ : is it all it’s cracked up to be?

Thoughts on the pros and cons of a stack of ‘best in class’ vendors vs an integrated platform.

The promise of the modern data stack is a seductive one: pick the best tools for each job, plug them together, and watch the insights flow.

And there’s no shortage of powerful, well-designed tools out there for extracting, transforming, loading, modelling, and visualising your data. But is the modular vision of analytics, combining ‘best in class’ tech, delivering the value it promises? What are the downsides of this approach, and what does the alternative look like?

Is your data stack really working as efficiently – or cost-effectively – as it could be?

From a distance, stitching together a stack looks elegant. In reality, it often leads to duplicated effort, finger-pointing, and creeping complexity. Let’s take a closer look.

Interoperability : does your stack really play nicely?

Modern tools offer APIs and docs – but seamless integration is rarely truly seamless. Dependencies build up quickly:

• Changing your your ETL processes?
  • Does your modelling tool need lots of updates?
  • Do these changes need to be synchronised?
  • Can that work happen simultaneously?
  • Can you do QA in both vendor platforms before pushing your changes?
• Updating a transformation?
  • Are downstream data model elements going to break?
  • What about the dashboards that are using them?
• Building new metrics?
  • Does my ETL vendor support these fields?
  • Do I need to make changes to my pipelines?
  • What about backdating this data?
  • Will my data team have to change their transforms before I can use these KPIs?

The ideas is a system that *seems* flexible in theory, but only at the macro level, i.e in terms of the core components you’re deploying. At the micro level the flexibility you need may well prove elusive (or very time-consuming and expensive to achieve.

A stack of vendors that were not designed to work together often resists change, and that’s a really big problem when the speed at which reporting can iterate is crucial. Your insights – and therefore your reporting – need to adapt at the speed of your marketing’s evolution.

What might I be missing out on?

Another question to think about is what you might be missing out on with multiple vendors? They may all do their jobs perfectly in isolation, with no problems, but if they were integrated at every stage, what could you achieve?

For example, an ETL platform that is directly integrated (via internal API) with the semantic layer can handle data caching logic without you having to think about. A data source update via ETL that tells the semantic layer which data sources have been refreshed, and for what time range. A reporting layer that is connected to the semantic layer via API means that the reports which have been affected by a data source update, or by a change to metric or dimension definitions, can be instantly refreshed.

At Bright Analytics, every component of our platform communicates instantly via internal API. This delivers a totally seamless flow of data source update messages, automated cache management, and automated report refreshes.

Costs : more tools, more cost, more surprise charges.

There’s a more complex financial reality to consider when multiple vendors are involved, with two specific problems that we see:

Multiple vendors means multiple contracts, likely over different terms. This can mean that you if you want to replace a component of your stack, you will find yourself paying for it until the contract expires and also paying for its replacement whilst that is onboarded and integrated. a solution you no longer need, whilst you onboard its replacement, meaning you’re paying double

When contracts aren’t aligned it can make strategic shifts hard – or just expensive – to execute, even when the business case is clear.

Different vendors will also have different pricing models to understand, and each component of your stack will potentially have its own tiered / variable pricing relating to users, rows, queries, connectors) to understand.

Does a change in the way you use one part of your stack cause a surprise increase in the costs you will pay for another component?

How much time is being spent trying to understand and mitigate costs when you just need to get on with your job of equipping your teams with accurate, insightful, actionable data?

Support : who owns the problems?

Multiple vendors means multiple support teams, different contacts, different portals, multiple;tiple people to explain the issues. If a metric breaks, a dashboard loads slowly, or a pipeline fails – where do you go first?

• Do you raise a ticket with your ELT vendor?
• Dig out the BI tool support knowledge base?
• Dig through Slack threads to find the last person who touched the dbt models?

In practice, even small changes can require co-ordination across 2–3 suppliers and internal teams. And if you’re not sure where the issue lies, you may end up managing the support process as much as the data.

With a unified service from an integrated platform, support issues and questions about expanding the scope of your data setup can be answered by a team who understand how all components work together. You should get a faster answer and fewer hand offs.

There’s a better way

At Bright Analytics, we’ve taken a different approach. Our platform is:

• Fully integrated : from data ingestion through to dashboards, everything just works and the data flows.
• Customisable : designed to easily reflect the unique shape of your business and reporting needs.
• Backed by a team : we’re here for the long term, working directly with you to evolve your setup.

We combine the power of the components in a modern stack with the simplicity of a unified platform – and none of the finger-pointing. One partner. One platform. Built around your needs.

More flexibility, better value, faster results.

If you’d like to hear more about Bright Analytics and how we work please book a demo