Data management in the Field
Capturing information in the field can be a difficult data management problem due to lack of connectivity. The sheer volume of data and paper based processes can also be concerns.
Recently we have been covering Arkady Maydanchik’s e-book, “Causes of data quality problems.” I covered some of his points on data management during a merger in this post last week.
This week, I’d like to cover some issues surrounding manual data entry, along with strategies for improving this process.
Where Manual Data Goes Wrong
As Maydanchik highlights, although we try to automate form filling, many databases are populated manually. “The most common form of data inaccuracy is that the person manually entering the data just makes a mistake.”
In the case of oil and gas services, this may be a paper ticket recording work from the field. For an Upstream company, this could be entering daily production into a database. In both examples, entering the wrong date or inputting a negative number can wreak havoc on the cleanliness and accuracy of a database.
One particular mistake that Maydanchik says many companies make is in the layout of the form, “Convoluted and inconvenient data entry forms often further complicate the data entry challenge…users tend to find the easiest way to complete the form, even if that means making deliberate mistakes.”
Error Proofing the Process
There is no perfect world in data management, but there are strategies that can mitigate many of the most common sources of error. On Maydanchik’s list, “fields are labelled and organized clearly, data entry repetitions are eliminated, and data is not required when it is not available or already forgotten.”
In addition, here are a few suggestions from our team that should be useful if you’re thinking about the best way to manage manual data entry:
- Proper form validation
If a well API number should be 12 numbers, the form filler can’t input a letter, or only 11 characters.
- Input masks
Phone numbers in the US are always three digits, another three digits, then four digits. An input mask can help ensure that this input is always the same
- Date validation
Inputting a date from the past or the future can be a significant date entry error. Forms can validate that the current day’s date is being input.
- Valid ranges
When inputting daily production values, the lowest possible number is zero, so a negative number would never make sense. Or, if yesterday’s production was 100 barrels, inputting 10,000 the next day is probably a mistake. Rules can be created to prevent entry of negative numbers or data variance of more than 50% in either direction.
- Duplicate data entry
If data needs to be entered in a database more than once, this process should be automated. Multiple points of entry also means multiple opportunities for error.
Data Management as a Priority
Concerns with manual data entry may not be at the top of your to-do list, but these values are really the starting point of your decision making process. If you can’t trust that field work data is correct or that production is accurate, then the decisions you make are suspect as well.
For more, read our blog on making transforming data into actionable information…