If you think Big Data is costly now—wait until you hear this one. In some cases, I suggest you double your investment in big data resources, especially when it comes to data scientists. Of course, you wouldn’t do this unless the value of your Big Data strategy could support it, which is why you must always do a value analysis before considering costs.
That said, if you have a tenable expected value of Big Data’s contribution to your strategy calculated at $227 million, you can spend $20m on your Big Data effort and get a very respectable return on your cost. With $2.5m per year you should be able to build a good team of 7-10 analytic leaders, managers, business experts, and data scientists.
With $5m though, you can build two good analytic teams. Now, it might sound absurd to spend twice as much as you think you should; however, spending twice on Big Data resources can be a very smart move for the savvy leader.
Two heads are better than one
There are a lot of benefits to having two teams do the same work, like: more brainpower for problem solving, diversity, and healthy competition. However, your biggest advantage to this approach is a sizeable decrease in your risk of failure (or an increase in your chance of success), from the quality of the analysis. This factor alone may bring your value proposition high enough to actually make it a better decision to double your team.
Here’s how that works. In your value analysis, you should always assign a probability that your strategy does not succeed. For Big Data efforts, I actually recommend an initial value lower than 50 percent—say 45 percent. This may not sit well with you, but the reality is that Big Data strategies are very risky propositions.
On a strategy with a potential value of $775m over the next 6 years, a 45 percent chance of success brings the expected value of your strategy down to $349m. However, even if you only raise that chance of success by five points to 50 percent, the expected value of your strategy jumps $38m to $387m.
Since the strategy is focused on Big Data, let’s say Big Data’s contribution to the overall strategy is a conservative 65%. In this case, the value of Big Data’s contribution jumps $24m from $227m (65% of $349m) to $251m (65% of $387). So spending another $2.4m per year on a second analytic team would still preserve your nice return.
By the way, a five point increase in your chance of success is very conservative—a reasonable increase would be about 20 percent, making this decision a no-brainer.
Convinced yet? If so, here are a couple of ways you can double up on your efforts: parallel research and pair-analytics.
Parallel research is when you have two separate analytic teams working on the same research question. For instance, you may have a hunch that the log data from your widget operations may be valuable to the luxury market, so you launch a Big Data strategy in that direction, and deploy two separate analytic teams to innovate on an analytic offering.
In this scenario, it’s important that you build two complete teams, and not just more data scientists working with the same business experts and analytic leadership. The diversity in management is just as important as the diversity in analysis.
With this configuration, I would recommend a mixed-methods research approach, with two qualitative analyses running side by side followed by one quantitative analysis. The two qualitative analyses will help answer a lot of upfront unknowns, and since it’s an interpretive style of research, it lends to a fascinating aggregate discovery when both efforts are combined.
The two parallel teams can communicate with each other; however, there shouldn’t be heavy collaboration. You want these efforts to be separate. The quantitative analysis will tie both qualitative analyses together, and put some confidence in the innovation with statistical rigor. This second phase can leverage the pair-analytics approach that we’ll discuss next.
Pair-analytics is a term I use to describe two data scientists working side by side on the same problem. The idea was inspired from pair-programming: a technique that agile software developers use to increase the quality of their code. In the same way, a pair of data scientists working closely together will produce analyses of a much higher quality than just one data scientist alone.
In this scenario, you’ll want all the data scientists working with the same group of business experts and leaders—there is no separation; it’s one big team. It’s also not critical that you have pairs of business experts, analytic managers, and leaders; however, it doesn’t hurt.
What is important though, is that your analytic leaders have familiarity with this teaming style. Analytics tend to like their space, so when you force them into a dyad there may be some personality issues to deal with.
The additional benefit of this approach is that it accelerates the timing on your innovation, which should accelerate your payback period. At the very least it brings the uncertainty down faster, as two heads can solve problems and answer questions faster than one.
A common question is, “If I double my cost will I cut my analysis time in half?” Unfortunately, the answer is no. Two heads work faster than one—but not twice as fast. Research puts the premium of having two heads at around 20 percent. So if one analyst could crank something out in a week, two might do it in about three days. You can use this to further strengthen your strategy’s value analysis if you use this approach.
When it comes to the resources on your Big Data strategy, two heads really are better than one—as long as your strategy’s value supports the investment. You can either run two analyses side by side and compare the results, you can fuse data scientists together to form power-twins, or you can do both. In all cases, you can expect to see the quality of your analysis jump, which will increase your chances of success. Take some time today to re-evaluate the value on your Big Data strategy—you might want to double down on your resources.