Counting Elephants - How to Solve Big Problems with Big Data

One ... two ... three ... four hundred ... five hundred thousand .... How many elephants can you count before it's too many (or too much)?

Counting is one of the first skills learned as a child.  Before addition and subtraction, the numerical building blocks of 1-2-3 are right there with the ABCs.  Whether it's your blessings or the dealer's cards or your money, counting comes in handy.  How many or how much is core to decision making.  How much money or how many resources do I have?   How many does the enemy have?

Counting to 10 or maybe 100 is easy, but as more needs to be counted, it becomes tedious and time intensive.  The practice loses its return on energy.  That's where math and probability and statistics come in.  Using what can be counted easily can be leveraged not only to count more but also to add value to the meaning of what is counted.

Enter the classic "what to wear" problem (poignant for math geeks).

Instead of laying out each combination and "counting" it, you know how many outfits you have.  This simple example can be exploited in far greater combinations...to the nth degree.

But what if you don't know how many shirts and pants you have?

Big Data Counting - The Next Generation

Counting is another excellent venue for exploring Big Data.  Just as math saves hours of manipulating closet items, so Big Data can help with Big Problems, providing greater choices and better decision capability as a result.

Let's look at a Big counting problem -

Just how many elephants are there in Africa? (and why does anyone care?)

A shocking increase in poaching has ripped down African elephant populations. In the past 10 years, the African elephant population has taken a dramatic hit with estimates that 12,000 plus a year have been slaughtered since 2006.

 ¡°The threat of local extinction feels very real. In October 2013, Elephants Without Borders flew a survey over a park where we had previously counted more than 2,000 elephants. We counted just 33 live elephants and 55 elephant carcasses. That is why this research is so important."

Dr. Mike Chase, director and founder, Elephants Without Borders.  http://www.elephantswithoutborders.org

Wildlife preservation is a delicate entente in the best of circumstances, but the lucrative draw of poaching in the myriad of African countries where they habitate has challenged several of its iconic epicenters.  Lions, rhinos and elephants are the majestic leaders of a rich wildlife pyramid whose dramatic loss crushes the whole ecologic system, including the native peoples that live and exist off the balance.

Poaching itself is lucrative.  The transit from Africa to Asia transforms ivory at $200 to over $2000.  Because of international standards outlawing this black market material, poaching profits only illicit activity and most dangerously - terrorism.

Except in South Africa

South Africa "suffers" from too many elephants.  Here the growing numbers continue to roam and forage as is their nature.  That means knocking over even the most sturdy of trees and stripping them of the best digestible leaves.  Just imagine an elephant walking through your yard or the neighborhood park taking down a couple trees that look tasty.  Imagine what a herd of 20-30 can do.  They just don't stand still either.  They keep on the move, journeying for miles in a day carving a pachyderm hurricane path.

In any amount, this is nature's process, culling the forest for new vegetation.  Their trails create natural fire breaks and they dig for water which other animals use.  But  where farm and urban sprawl encroach this roaming territory, it quickly becomes man versus animal.  The number of touch points are growing too.  The nature of elephants - their survival - is roaming.  Their legendary memory too has them cross paths where man's development has erased the past.

To attempt that delicate balance, game parks in South Africa have taken to birth control and water management methods in order to keep their numbers in check.

So What Numbers Are We Talking?

Anyone who has tried to count children at a birthday party or getting all students back into a classroom after recess knows the challenges of counting live bodies. Counting crowds is actually a science. And Science isn't about Knowing so much as Getting a Good Estimate.  Here's how they counted President Obama's inauguration crowd.

Although a several ton elephant is noticeably slower and harder to miss, expand the search over the wildernesses of half a continent dissected into several countries, some war torn, and accurate counting is hard to imagine.  But someone is trying.

Not Just Throwing Money at the Issue

That effort is the Great Elephant Census, the largest pan Africa aerial survey since the 1970s, and it's backed by one of the world's smart guy-in-the-room icons - Microsoft co-founder Paul Allen.  Not only does this count have deep pockets, it also has expert guidance.

About the Count

The Great Elephant Census is applying a strategic, consistent approach to counting elephants in numerous countries in varying climate and terrain, with an integrated audit program in situ.

The Great Elephant Census is designed to provide accurate and up-to-date data about the number and distribution of African elephants by using standardized aerial surveys of tens of hundreds of thousands of square miles. Dozens of researchers flying in small planes will capture comprehensive observational data of elephants and elephant carcasses. Our standardized method of data collection, which is validated by an independent TAT advisor ensures all data is impartial and accurate.

It's somewhat like counting the crowds for President Obama's inauguration.  Even with such meticulous effort though, most elephant accounting is predicated on "known" and "estimated" numbers.

But ... Back to Big Data

So that's how the experts are counting elephants.  Let's explore counting elephants instead with a Big Data lens.  An elephant census doesn't have to solely be tallying head counts, albeit a magnificent head with flowing ears and strong tusks.  The count can be created through a variety of volumes of data that exists already and grows by the minute.  In a Big Data Elephant Census, information is created by the community and also serves the communities in return.

Big Data Elephant Census begins with a data lake of information collected from the prevalent sources:  cell phone usage, transactional data, weather, heat signature, game warden activity/reports, international shipping and markets, and of course, social media.  Big Data ingests the volume, velocity and variety of the data to look for patterns that emerge.  Like trying to count moving children, Big Data can exploit information that is too complex for "naked" human observation.  Like picking outfits from the mathematically derived wardrobe, Big Data Elephant Census provides an answer to how many elephants as it elicits the holistic picture of what that means.

THE POINT OF COUNTING ELEPHANTS NOT TO KNOW HOW MANY ELEPHANTS THERE ARE.  THE POINT OF COUNTING ELEPHANTS IS TO LEARN THE ELEPHANT POPULATION EFFECTS ON OUR LIVES  - DESIRABLE AND UNDESIRABLE.

The point of counting elephants is not just to know how many elephants there are; we want to know all the factors that evolve in the elephant environment.  How does the diverse animal and vegetation habitat ebb and flow with the tramping of elephant feet?  How are indigenous and foreign humans influencing and being influenced by the elephant footprint? (har har) How are poaching and anti-poaching efforts impacting the community as well as the elephants?  How are farming and native livelihoods affecting and being affected?  What other passive economic factors, weather, and politics shift accordingly?

CONSERVATION IS NOT A STASIS ...

So let's stop trying to capture a picture and instead focus on what the flow is. Pulling Big Data elephant count from a volume, velocity and variety of data sources articulates how the system (man and animal) manifests.  Instead of trying to chase the right amount of elephants, Big Data Elephant Counts observe the evolving energy to find the signals in the noise.  Gentle shifts or environmental shocks are recorded in situ with all the elements and players.  Big Data Elephant Counting is less reactive and is both predictive as well as evolving - like the trails we all carve through the forest.

Article Author: Colette Grail

Author for Datafloq