Custom Software Apps & SharePoint Consulting

PNEC 2014 Recap: Dealing With The ‘Big Data’ Crisis

I attended the 18th International Conference on Petroleum Data Integration, Information and Data Management (commonly known as the “PNEC” conference) on May 20-22, 2014.  As I reflected on the various papers and talks, it was clear that “Big Data” is both a significant challenge and an amazing opportunity for upstream oil and gas.  I was inspired by a talk on the topic of Big Data and it got me thinking about how you actually go about unlocking the power of Big Data.

3 signs that big data is trapped

  1. Too much data
    Consumers of data in the business typically lack the technical skill set required for big data integration and analysis.  In order to effectively mine your data, you must be able to assemble the data in a readily consumable format.  Due to the volume of data being generated daily in a modern upstream oil and gas operation, there is so much data any one business user won’t know what exists, where it is or how to get it.
  2. Too many repositories
    As a result of the volume of data and the specialization of various departments (e.g., G&G, drilling, production, accounting, etc.) each department typically has its own preferred system of record.  While having specialized systems is beneficial, if not outright required, it results in “stovepipe systems” or data silos.  Additionally, because common business metadata is logically shared between departments, data is duplicated and data quality suffers.  With no single version of the truth, it is difficult to ascertain what a well name is, what the spud date was or in many cases what even a “well” is (e.g., a completion, a wellbore, a wellbore completion, etc.).
  3. Unable to analyze effectively (can’t ask the “What if…” questions)
    Even if you have all of your data in one place, making it accessible to the business users in an easily consumable format requires specialized Business Intelligence software.  A modern BI platform allows you to put the power in the business users’ hands to get the insights they need without requiring IT to spoon-feed the information.  IT’s responsibility should be to put a solid and extensible platform in place to empower the creativity and natural curiosity of the subject matter experts within the business.

How do you unlock the value of your information assets?

  1. Identify the systems of record
    To get started, you have to know the systems in which the data resides and how it gets into those systems. This is a discovery process that is required because even if you think you know where everything is, more than likely you don’t have the full picture. Keep in mind that vast quantities of data may be locked-up in spreadsheet files on a user’s desktop or even on paper in a remote field office!
  2. Create an integrated repository
    Once you’ve identified the systems of record, the next step is to build a “Master Data Management” (MDM) solution that aggregates and synchronizes all of the key business metadata.  In addition, a comprehensive data warehouse should be layered on top of the MDM solution to provide a single location against which aggregate business intelligence analysis may be performed.  It is important to leverage standards when embarking on MDM and data warehousing projects (I talk about that more in another blog post).
  3. Use a modern Business Intelligence tool
    Excel remains the Swiss Army Knife of business software, but does not provide the powerful data visualization capabilities of a modern BI tool.  There are many great tools in the BI software space today.  Applications such as Tableau, Spotfire, Power BI and many others put business users in the driver’s seat with powerful yet easy-to-use data visualization features.  However, by themselves the usefulness of these tools is highly limited unless you’ve done your homework to build out MDM and a data warehouse for your business.  Ultimately, you want the business to be agile, i.e., able to quickly adapt and change direction based on insights from the organization’s collective data.  When the business users can ask the “what if” questions and easily get answers, then you know you’ve reached BI Zen.

What does all of this mean in the end?

The ability to efficiently aggregate and analyze data is the goal of the “big data” movement. Buzzwords aside, ask yourself if you are extracting all of the insights that your data can provide. If not, then it’s time to Frac Your Data and Produce Insight. The energy software experts at Entrance have the experience and skills you need to unleash your trapped data.

Share this post with your friends

Skip to content