As a small business owner, I know the importance of understanding costs before starting projects. For example, you may have the best marketing system since sliced bread, but I’m still not going to spend $15K in one go. The risk is too high.
My clients, on the other hand, are all much larger than my company. They generally range between $50 million and $500 million in revenue. (Although we’ve done a lot of departmental level work for multi-billion dollar companies—and their finance folks.) But even they want to know what a business intelligence project is going to cost before they invest. (Finance executives tend to be risk averse. They hope for a return, but mostly they want to know the size of the hit if it doesn’t work out).
There are too many parts to business intelligence to cost them out in detail in one post. But based on the type of work we do at Red Three, I’ve put together basic estimates.
Our Business Intelligence Consulting Rates
Our rates range from $150 to $275 per hour. So, for budget purposes, a rate of $1,500 per day is a good number to work with. Those rates, combined other estimates, make us either frighteningly expensive or the best deal ever.
For small companies, $20,000 for a bunch of reports can induce an attack of angina. We understand that. And we suggest you look for a good contractor or stick to manual processes.
For mid-sized companies, we’re a great deal. Few of our competitors with similar pricing have our accounting and business intelligence expertise. And many competitors price at much higher rates than us. For example, one of our clients found that our estimates (which we considered generous) where less than 10% of what a big consulting firm had charged them for a similar project.
While there are many different kinds of business intelligence projects, I’m going to focus this post on estimating costs of three particular types of work which represent a large part of what we do:
- Cost of SSRS Reporting (i.e. writing reports over existing data)
- Cost of building a data mart
- Cost of building a data warehouse.
Note: All our cost estimates below are for consulting time only. We don’t include hardware and software costs for two reasons:
- We use Microsoft stack—including Excel, SSRS (SQL Server Reporting Services) and other SQL Server related tools. If you’re like most clients, you already own these technologies.
- If you do have to invest in additional software, it typically represents less than 10% of any project expense.
Another note: We don’t estimate data validation costs because they vary widely with the quality and size of the data.
You can certainly ask us to review your data. And we always do our best to make sure the numbers are consistent. But only you know your data. And your people need to have the time to make sure things are working as expected.
We do a lot of this kind of work. As you’d expect, reports vary from simple vendor listings (for which you probably wouldn’t hire us) to complex project profitability statements (including current project AP and AR status). Over that range, reports can take anywhere from one day to five days to complete, with the average taking about two days (or $3,000 of work). Ten reports take 20 days or $30,000 of work.
You might think you could find a report developer for a lower rate, and you probably could. But the reports we work on usually require sophisticated knowledge of SQL. We’re not just making pretty invoices. So, our developers need to not only understand SSRS, but also database structures, and, critically, the ins and outs of accounting and other ERP data.
There’s one more factor to consider. You might have a high RQ ratio (multiple reports covering basically the same set of data). This means everyone in your company—despite your best efforts—wants (and gets) a slightly different version of the same report. As an analogy, if they were all getting ice cream, they’d all get vanilla—but one person wants chocolate sprinkles, another wants M&Ms on top and a third wants only blue M&Ms. In a high RQ ratio situation, we can sometimes group similar reports into different sets. Then, the initial report might take four days but each additional version will only take one.
So, let’s say that instead of 10 individual reports, you have two groups of five:
Note: We never estimate assuming high RQ ratios until we’ve actually seen the reports.
So for ten reports, you can expect costs between $21,000 and $30,000, depending on report differences and similarities.
Another note: We write reports in SSRS. But these estimates also held true for Crystal and other reporting tools we used to use.
We recently wrote a post on how much a data mart costs, which we suggest you look at. As a reminder, the purpose of a data mart is to make reporting and analysis easier. The database developer selects, merges and calculates various data so that report writers and business analysts can get to work asking questions and finding answers.
Generally, our time estimate for creating a data mart for one subject area (e.g. financial reporting or project profitability) is between eight and 15 days. The variance will depend on several factors:
- How organized the data is in the source system. Again, we’re not integrating multiple systems in a data mart project. The data’s already in one place. But it can still vary greatly in how well it works.
- How many dimensions the data mart needs. If your financial reporting data mart is limited to account and business unit, you probably won’t have performance issues. But as we add more dimensions, we have to spend more time optimizing the data set. Data marts are made to optimize reporting. If you end up putting almost every field from your original system into your data mart, you need to rethink your strategy.
- The size of the data set. This is pretty obvious—a larger sets means longer load times and more processing issues.
In any case, you can still apply our $1,500 rate to the 8-15 day estimate and plan on $12,000 to $22,5000 for a basic data mart. In practice, most data mart projects are combined with reporting projects. And once a data mart is built, reports are more likely to take one day (not two days) to write.
When we use the term data warehouse, we mean a data warehouse over a particular area of your companies data, such as financials, projects or sales (or perhaps all three—which we’ve done before). Once upon a time, people thought of a data warehouse as combining ALL data in one place. We’ve never been involved in such a project (and haven’t heard of too much success in that area either.)
When estimating the cost of building a data warehouse, you need to consider three things:
- The nature of the data repository
- Time needed to extract, transform and load the data (i.e. getting data into the repository from other systems)
- Time needed to create reports over the data.
The Data Repository
Again, we begin with the caveat that most of our work is based in financials, and these estimates are based on financial data—from journal entries to sales to product/ customer/ project profitability. (HR data has a different set of challenges. We’ve worked with HR data, but it’s no longer our focus.)
A data repository is a single place where all your data sits nicely—central and well organized. But it may not be optimized for reporting yet. (That’s the job of the data mart.)
Sometimes, you can use your general ledger as a kind of poor mans’ data warehouse, but let’s say we really do need to build a new repository. We generally allocated 6-10 days for this process or $9,000-$15,000. This includes basic design as well as specifications for loading data into the system
Extract, Transform, Load (ETL)
The reason for building a data warehouse is that your data is scattered across many different systems in your company. You need to get it out of those systems (extract), manipulate it to a standard format (transform) and place it in your data warehouse (load). The size of the job will depend on a number of factors:
- How many systems need to be linked?
- How many kinds of data need to be extracted from each system?
- How many mappings need to be built?
- How long will the final transactional transformation take?
Let’s take an example: We want to create a repository of all our financial data. We have three accounting systems around the world, in the U.S., Canada and Sweden. We want to track data at the account/customer/project level.
Here’s a graphic to help illustrate this example:
And here’s what a basic estimate would look like:
A few notes on the above chart:
- Extracting data from each system is pretty much a repeating cost. Each system has its own peculiarities so there’s no economy of scale.
- We increased the time needed to extract Sweden’s data because the data is in Swedish and needs to be translated.
- Mapping does get cheaper with scale. If you create a way to map Esso Canada to Exxon U.S., you can use the same methodology to map Esso Sweden to Exxon U.S.
- Transactions always take the most amount of time. The dollars have to be exactly right.
- These estimates including a fair amount of testing support. However, your users MUST be available to map and check data.
- Data can come from any system, but we do all mapping and transforming with Microsoft tools. We do most of our work in T-SQL, but sometimes we use SQL Server Master Data Services (SSMDS) and/or SQL Server Integration Services (SSIS).
Pulling it all together, you can see that integrating one system into a data warehouse will take between 15 and 21 days or $22,500 and $31,500. You then have to consider reporting and/or data mart costs, which we’ve already covered, above.
So these are the ballpark numbers we provide to clients and prospects when they ask us, “How much does business intelligence cost?” If you’d like to get a customized estimate for any of these services, feel free to contact us.