How do you integrate oil and gas regulatory data from multiple US states?

Integrating US oil and gas regulatory data means combining state regulators that differ in what they publish, how you access it, and what one record represents. Texas reports oil production by lease, not by well. Oklahoma's production comes from its Tax Commission, not its oil and gas regulator. Start with a source registry, then map each state into one model.

This guide is for data leads, product owners, and engineering teams at oil and gas data businesses, mineral and royalty companies, and operators working across several states. The usual trigger is expansion: the integration that works for one state falls apart at the second, because the second state doesn't just use different files. It reports different things.

This article is about the regulators and their data: who publishes what, how to get it, and how to reconcile it. For the pipeline mechanics, see our guide to oil and gas ETL pipelines. For checking the results, see our oil and gas data quality best practices.

Who publishes oil and gas regulatory data in the US?

Oil and gas regulatory data in the US is published mainly by individual state regulators, with separate federal systems for wells on federal and tribal land. There is no single national source.

The National Petroleum Council's 2011 paper on regulatory data management counted 38 oil- and gas-producing states or Native American nations, each with its own regulatory regime. Efforts to unify them have been partial:

• RBDMS. The Ground Water Protection Council's Risk Based Data Management System, developed with the US Department of Energy, is used by more than 20 states for permitting, completions, and production reporting. Shared software doesn't mean shared data, though: each state configures and publishes its own.
• National Oil and Gas Gateway. In 2017, EIA, the Ground Water Protection Council, and DOE launched a public site with well-level data from 10 states, including Colorado, Oklahoma, and Utah. It covered 10 states and did not include Texas, the largest producer.

Responsibilities are also split within some states. In Oklahoma, the Oklahoma Corporation Commission publishes well, completion, and injection data. Production volumes come from the Oklahoma Tax Commission, which is the state's official recordkeeper for production. An Oklahoma integration therefore needs two sources with two different identifiers.

How do you access each state's oil and gas data?

Each state offers its own mix of access methods, so access has to be planned per state before any code is written. Here is how six states compare:

Three rules for planning access:

1. Prefer bulk files over scraping web queries. Bulk files are what the regulators designed for reuse, they're more stable, and they put less load on public systems.
2. Budget for paid and request-based sources. North Dakota's subscriptions and Oklahoma's production history request form have costs and lead times.
3. Read and record each source's terms and contacts. Where terms are unclear, ask the regulator. The Texas Railroad Commission, for example, lists contacts for questions about its online queries.

Why doesn't "production" mean the same thing in every state?

"Production" doesn't mean the same thing across states because each state reports it at a different level and under different rules. Combining records without accounting for that produces totals that look right and aren't.

The differences are documented by the regulators themselves:

• Texas reports by lease. The Railroad Commission's Production Data Query FAQ says production is reported by lease rather than by individual well. An oil lease can include many wells, while a gas lease has one gas well. Crude oil figures also exclude condensate from gas wells.
• Oklahoma reports by Production Unit Number. The Tax Commission tracks volumes by lease using a Production Unit Number (PUN) that it assigns. That number is different from the API well number used by the Corporation Commission.
• Pennsylvania reports unconventional wells monthly, by well. The Department of Environmental Protection's reporting guide requires a report for each well within 45 days of each month's close, on a wellhead basis where available.

Timing differs too. Texas operators file by the end of the following month, so the online data runs about two months behind. The Railroad Commission says records are mostly complete after about six months.

The practical rule: store production at the grain each state reports it. If you need well-level estimates for lease-reported states, calculate them in a separate, clearly labeled allocation step. Never overwrite the reported figures.

How do you match operators across states and over time?

Match operators with a master operator table that maps each state's operator identifiers to one company record, with effective dates. Operator names and numbers are less stable than they look.

Two examples show why:

• Texas operator numbers can be reused. The Railroad Commission's P-5 organization data guide says each organization gets a six-digit number. A number can be deleted and later reassigned to a different organization, and the data includes a flag for numbers used by more than one organization.
• Oklahoma ownership changes lag in tax data. IHS Markit reported that Tax Commission records were often slow to reflect well transfers, and sometimes swapped operator and purchaser names. Switching to Corporation Commission transfer data let the firm correct the current operator on more than 15,000 wells.

Rules for an operator master:

1. Key operators by state identifier and effective date range, not by name.
2. Treat names as attributes. Normalize punctuation and suffixes, but confirm merges manually.
3. For each attribute, pick the source that is most timely. In Oklahoma, that's the Corporation Commission for current operator and the Tax Commission for volumes.
4. Keep the history, so past production stays tied to the operator of record at the time.

How do you reconcile state and federal records for the same well?

Reconcile state and federal records by deciding, attribute by attribute, which system is authoritative, and by flagging conflicts instead of silently picking one. For wells on federal land, the same well can exist in a state system and in several federal ones.

Federal records have known gaps. A 2021 Government Accountability Office report (GAO-21-209), summarized by Taxpayers for Common Sense, found that:

• The Bureau of Land Management's well system (AFMSS), its lease system, and ONRR's production system are outdated and don't communicate well.
• Earlier reviews found 20% of AFMSS wells had no matching bond number in the lease system, and 40% had a different status than in ONRR's system.
• BLM staff often relied on state regulators' data instead.
• ONRR spent more than 22,750 hours a year converting and correcting data.

For federal wells, a reasonable default is this. Use state data for well header and status, ONRR data for federal royalty volumes, and BLM's AFMSS public reports for federal permits and idle wells. Record which source supplied each value.

How do you keep up when state regulators change their systems?

Keep up by treating every state source as something that will change, and by monitoring each one for structural changes, not just new data. Regulators rename themselves, retire formats, and replace systems on their own schedules.

Recent examples:

• Colorado's regulator changed its name. The Colorado Oil and Gas Conservation Commission became the Energy and Carbon Management Commission on July 1, 2023, with expanded authority, according to Beatty & Wozniak. Names, domains, and document references changed with it.
• Texas retired a file format. The Railroad Commission replaced its R-3 EBCDIC file with JSON in 2025.
• California's system is still evolving. CalGEM describes WellSTAR as a database under ongoing development.
• Oklahoma revises published files. The Corporation Commission notes that its 2020 injection volume file is incomplete and subject to change.

What is a step-by-step playbook for integrating a new state?

A repeatable playbook turns each new state into a defined project instead of an open-ended investigation. These are the steps:

1. Register the source. Record the regulator, every dataset needed, URLs, access method, cost, terms, contact, update schedule, and revision behavior.
2. Map responsibilities. Note whether wells, production, and permits come from the same regulator or from different ones, as in Oklahoma.
3. Document the grain. Record whether production is reported by well, lease, unit, or PUN, and what it includes or excludes, such as condensate.
4. Map the identifiers. Record the well identifier format, operator identifiers, and lease or unit identifiers, and build the crosswalks to your model. PPDM's US Well Number Standard is the reference for wells.
5. Choose the system of record per attribute. Especially decide where state and federal sources, or two state sources, overlap.
6. Write acceptance tests. Reconcile your loaded totals against the regulator's published totals before the state goes live.
7. Monitor for change. Check schemas, file names, and arrival times on every run, and review each source's registry entry at least once a year.

Steps 1 to 5 are research, not code. Skipping them is the most common reason a second state takes longer than the first.

Proof: integrating all 50 states for Peak ETL

Peak ETL, an oil and gas data business in Colorado Springs, needed regulatory data from every producing state in one product. Its customers, landmen and speculators, had been piecing that data together from courthouses and state websites by hand, or paying six figures for enterprise tools.

6sense HQ built Spud Wire, a map-first platform that brings well, production, land, and regulatory data from all 50 US states together. Sources arrive through SFTP, scraping, Excel, and CSV, and the architecture absorbs each state's format without rebuilding the system.

Jason Holden, Owner of Peak ETL, said: "6sense HQ delivered the working platform and stayed responsive as requirements changed."

When shouldn't you integrate state regulatory data yourself?

Don't integrate state data yourself if a commercial data provider already covers the states, fields, and freshness you need, and the integrated data isn't your product. Integrating many regulators is ongoing work, not a one-time build. Every source keeps changing.

Integrate it yourself if:

• The integrated data is what you sell.
• You need a state, dataset, or level of detail that commercial providers don't offer.
• You must combine regulatory data with your own operational or land data under one model.
• You need control over how reported and allocated values are labeled.

For a structured way to decide, see our build vs. buy guide for energy data software.

When you shouldn't work with 6sense HQ: if a data subscription meets your needs, buy it and skip us. We're also not a data reseller. We build and maintain integrations that you own, and we usually start with one state or one data type to prove the model before expanding.

The bottom line

Integrating US state oil and gas regulatory data is mostly a research problem before it is an engineering one. Each state has its own regulators, access methods, reporting grain, and identifiers, and some split those across bodies, as Oklahoma does. Register every source, store production at the grain it is reported, maintain operator and well crosswalks with history, choose a system of record per attribute, and monitor each source for change. With that discipline, each new state becomes a defined project rather than a rebuild.

About the author: AKM Ahsan is CTO of 6sense HQ, a software engineering partner for energy, mobility, and operations-heavy companies. 6sense HQ built the 50-state data platform behind Peak ETL's Spud Wire. LinkedIn

Adding your next state, or stuck on the second one? Book a 30-minute technical review with a senior engineer and get a straight read on what to tackle first, whether or not you work with us. See how we approach energy data engineering.