Guidance and results for the data-rich, yet information-poor.

Category: Articles

The Fallacy of Big Data

It's going to take more than a mess of information to solve real business problems.

It’s going to take more than a mess of information to solve real business problems.

Everybody’s excited about “big data.” Organizations all over the world are leaping on board the Big Data Bandwagon, and are eager to see what it can do.

There’s just one problem. “Big data” isn’t exactly what you think it is. There are a lot of problems with the term, and those problems lead to misconceptions that keep you from making the most of your organization’s data.

“Big Data” is a marketing term.

There’s nothing new about predictive analytics. The science has been around for decades.

Big data has just been the particularly successful marketing term that has catapulted predictive analytics into the consciousness of the mainstream. But sooner or later, “big data” will start to lose steam. What will happen then?

Marketers will still need to sell predictive analytics software. So they’ll make up another term. The new term might be just as successful or it might fall flat, but it will still be a new name for the same science.

You need to know this, because it will keep you from making silly decisions. You will not, for example, leap to buy the most expensive “big data” software package on the market when you know that people have been performing many of the same functions in boring old Microsoft Excel for years and years.

“Big Data” assumes that bigger is better.

In truth, bigger isn’t always better, and “big data” can often mean “having more data than you know what to do with.” Many organizations need to master small data before they start messing around with big data…and many of your insights are going to come from sample sizes that represent only a small portion of the data that your organization has been collecting.

Stop worrying so much about big. It’s more important to shift your mindset about data. Gathering more and more data can’t help you until you’ve started asking the right questions. Namely: what problem are we attempting to solve by delving into this data? Everything else has to flow from that mindset.

Don’t ignore your data. But don’t romanticize it, either.

Yes, your data does have a lot to tell you. TMA wouldn’t be here if it didn’t.

You just can’t afford to get swept up into the hype. Data is just a tool, a tool that you will hopefully use to discover solutions to some of the problems that your organization is facing.

Data, and what you can do with it, is not magic. It is math. And when you recognize that, you can approach it as a mathematician would, which means ignoring all the hype. Instead, get laser focused on creating a data analytics program that will be truly useful to you.

Will TMA’s Training Help You Launch Your Data Mining Career?


Many of those seeking training through TMA’s courses are doing so because they have their eyes on a new career in data mining and predictive analytics. Naturally, they want to know if employers will see the value in the completion of the course.

It would be simple to offer a “yes” or “no” answer to this question. However, it’s probably more useful to take an in-depth look at what TMA training courses do for students.

TMA’s courses are not just seminars. They are also intensives…concentrated, aggressive classes designed to give you real world experience in the art and science of enhancing business performance. A great deal of this education is offered before you ever set foot in the classroom. TMA sends you 10 modules that will introduce you to the core modeling concepts that you need to know in order to get the most out of the class. These modules take 15 to 40 minutes to complete. If you don’t do them you’ll be lost by the time you reach the classroom! If you do take the time to complete them, however, you’ll be well-positioned to make yourself employable.

Your attractiveness as a business intelligence employee will not necessarily come from the certificate you receive at the end of the class, however. Nor will the course itself turn you into “top talent” all by itself. After all, taking a few lessons with a pro golfer does not usually turn the student into a pro golfer. That takes time, practice, and dedication that can’t happen in a matter of days.

Instead, you’ll stand out because of the way that you converse with employers about data mining issues. You’ll find that you’ll be able to offer an approach that is very different (and far more insightful) than the one that will be offered by most of your competitors.

That’s because TMA will take you beyond software skills. You’ll learn project planning so that you can produce actionable insights for your organization. You’ll also be equipped to navigate some of the inevitable political conflicts that crop up whenever anyone launches a new data mining project.

That means the answers that you will be offering during your next job interview will be far more interesting and convincing than the answers offered by the typical candidate. For example, a common interview question is: “How do you make your models more accurate?”

You will be able to offer this diplomatic reply: “Model accuracy is really only important if the project is defined and designed appropriately. Otherwise, you’ve just created a faster car that’s running in the wrong direction.”

The other answers you offer would then continue to flow from this unique perspective, because by the time you’ve completed TMA”s training courses you should be well-equipped to think about and solve complex business problems.

Data Mining: How do You Choose the Right Software?

Software. It's only as good as the analyst using it.

“How do you choose the right software for data mining and predictive analytics?” This is one of the first questions that people ask any of the experts at TMA.

Asking this question first is putting the cart before the horse, but it is a valid question to ask. After all, it’s wise to make sure that your company is making a right-sized investment when you look for tools to support your predictive analytics efforts.

TMA is vendor neutral. There is no such thing as the “one true software package.” However, there are principles that you can follow which will ultimately lead you to the right software once it is time to begin shopping for one.

Project strategy always comes first.

The very first step in the software selection process is making sure that you are ready to choose a software package in the first place. That means that you’ve already gone through the project planning process. You’ve focused on a real business problem. You understand which of the many data mining techniques are good fits for solving the problem. You’ve cleaned up your data and understand what data sets will be most useful.

In short, you want to make sure that you’re not doing “big data for the sake of big data.” You know where you want to drive this car.

Run test problems.

Once you’ve defined the problem it’s time to do some test problems. Launch a pilot project with open source software to see if you’re on the right track in how you or your team have chosen to approach this problem.

This will give you a much more educated picture of this project’s actual software requirements, and may give you some ideas about future projects, too. This will also tell you whether additional software is even required. It’s possible you might derive a great answer from open source software. If you do, your right-sized investment could be “zero dollars and zero cents.”

Remember the most important rule of choosing predictive analytics software.

Finally, guard yourself against overzealous software sales reps by remembering an important truth. The tool is never better than the analyst using it. If an analyst can’t select which techniques and algorithms apply best to the problem then there isn’t a software package in the world capable of truly helping your business.

Ready to become a better analyst? Sign up for a TMA training course now, and get ready to create an outstanding data strategy for your organization!

The 5 Biggest Lies About Data Mining

biggestliesMore and more people are catching on to the idea that there is more to data mining than the “big data hype” would suggest. But there are still 5 very big, persistent lies (or myths, if you prefer) that plague conversations about data mining or predictive analytics.

If your organization can move past these lies you will be able to reap the benefits of data mining and predictive analytics as never before. Each lie holds you back from doing all you can, so it’s important to understand them and to debunk them, one by one.

1. All Companies Use Data the Same Way

A lot of people think “Amazon” or “Netflix” when they think about predictive analytics. It’s certainly true that predictive analytics can be used to offer individual recommendations to customers, and it is true that in certain industries that capability can increase sales.

But there are many other ways to apply data mining and predictive analytics. If you want to increase sales you can also use it to place products in an advantageous way. You can use it to adjust the wording or the timing of a sales letter to get a very profitable incremental improvement. If you want to decrease waste you can use it to decide exactly how many widgets to order. If you want to decrease fraud you can use its ability to target useful outliers to help you find the fraudulent and bring them to justice.

Every organization has different needs, different, pressing problems, and even different stores of data. All of that will help determine what the best and most profitable projects will be for your company.

2. Choosing the Right Software Package is the Most Important Step

Many organizations approach data mining backwards. They say, “Well, data mining is a big thing now. We should probably be doing some of that.” Then they send someone off to decide which software package to buy.

The result is an expensive purchase and an expensive implementation that goes absolutely nowhere.

Instead, the company should say something like this: “We really need to figure out why sales plummet in April and what we can do about it. Data analytics can help with that.”

The company can then turn to one of two resources to solve this business problem: any open-source data mining software, which is free, or Microsoft Excel, which companies already have. From there it’s a matter of preparing the data and setting up the project in a way that ensures that it is capable of actually asking the question.

3. You Need to Hire a Data Scientist

TMA has covered this misconception before. A data scientist is usually the last person you need.

Data scientists create algorithms. Many of those algorithms already exist inside of Excel or other programs. 90% of the time nobody needs a better algorithm. They need a better approach for using the algorithms they already have.

Does that mean there’s no value in developing more powerful algorithms? No, of course not. That’s like saying there’s n o value in creating a better truck. Most companies, however, don’t directly hire engineers to create better trucks. They buy the best trucks on the market, then look for qualified people to drive those trucks.

4. Data Mining is Only for Huge Organizations

Organizations of any shape and size can use data mining to some degree, depending on what data they are collecting and what they are trying to do.

Obviously size and budget do open up the number of tools you can buy. If you absolutely think it’s necessary to track the movement of shoppers around your store to solve your business problem then it helps to have the budget to buy the software and hardware that can do that. Often, however, shopping patterns can as easily be determined by digging into the sales records of any store.

Data mining is most often effective at creating incremental change. Small changes can often produce big results, and small changes can stack to create even bigger results. A 1% increase in the effectiveness of your sales letter campaign doesn’t sound like much until you think about just how many people you’re mailing, just how many new customers that 1% increase would represent and just how much more revenue will be generated. What would happen if you could increase that number by 1% again? Big things.

It’s not the size of the organization but the needs of the organization that matter.

5. You’ll need four more years of training to ever make this happen.

Training is absolutely key to learning how to use data mining correctly. However, it hardly requires anyone in your organization to go pursue a second 4 year degree. A few weeks of seminar training are often all that is necessary to fully realize the potential that data mining and predictive analytics hold for your organization.

Ready to get started? Register for TMA’s free webinar today, or sign up for any one of TMA’s more intensive training courses.

Columbus Zoo: Doing Data Right

ZooIt’s one thing to talk of actionable business intelligence in the abstract. But it’s also great to spot an example of an organization that is truly approaching their data in a strategic way.

Data Informed ran an extensive interview with executives at Columbia Zoo to talk about some of the focused data analytics projects that they’ve already developed and run at the zoo. Some early successes include:

The zoo using BI to determine whether or not new exhibits are likely to be successful.

Using data analytics to make decisions about under-performing food and retail stands throughout the park, reducing waste and increasing profits.

Using data analytics to make an effort to foster more effective interactions with the donor base.

Of course, TMA has shared many data success stories on this blog. But this interview was very telling as it covered the thought processes behind the success stories, and that is worth highlighting.

For example:

We started to look at what we didn’t know about ourselves…and said, “We have data problems. We don’t really know enough to be efficient at what we are doing or to make really good, fact-based decisions.


We had a couple of key projects that we knew could make an impact right away…We knew we had a few targets we could fix right away and make a big impact with, and that in itself gave us a chance to kind of snowball the system in to where other departments looked and said, “Oh that’s what you can do with that, and now we can visualize that and see that, we can touch these reports. And, by the way, can we have this and this and this and this?”

This process of discovering those first key projects and targets are really key to creating a truly data-driven organization. Do what matters most first. Get successful, create results, and move on to the next one. Waste zero time on hype–don’t create new problems trying to solve old ones. This is excellent strategic thinking at work.

If you’d like to learn how to develop this kind of business intelligence strategy for your own business then it’s time to think about laying the groundwork that will put you in the right mindset to do just that. Get started today by registering for TMA’s free webinar, Data Mining: Failure to Launch. By the time you’re done you should be well on your way towards identifying those key targets and issues in your business environment.