Version 4 Data Pipeline Release
- Release:
- Tags: API, Data Release, Improvement, Map, Marine Manager, Vessel Viewer
Version 4 of data pipeline now includes AIS, VMS, object detection and vessel identity data
Global Fishing Watch has launched an improved version — version 4 — of our automatic identification system (AIS), vessel identity, object detection and vessel monitoring system (VMS) data pipelines. The data pipeline is the series of processes that we use to convert the raw satellite AIS ingested data, vessel registry, imaginary and country vessel monitoring system raw data to the data products, such as machine learning estimates of fishing activity, identity data that combines merged external registry data, and related vessel activity at sea that is then applied by users.
This release marks a significant milestone: for the first time, we are updating all of our data pipelines simultaneously. Unlike our Version 3 release, which focused exclusively on AIS (details here), this update covers our entire data ecosystem. To help you navigate these changes, we’ve provided a breakdown below of exactly what data is included in each specific pipeline and a summary of the impact in version 4.
| Data Pipeline | Data included | Version 4 impact |
|---|---|---|
| Automatic identification system |
| There are no major changes to the data values or logic in this release. |
| Vessel Identity |
| This update introduces data maintenance fixes. For a comprehensive list of the specific corrections, please see the summary below. |
| Object detection |
| There are no major changes to the data values or logic in this release. |
| Vessel monitoring system - Country specific data |
| The new unified VMS architecture accelerates the onboarding of country-specific data while resolving legacy data issues for more reliable data. Please see the details below. |
Looking for this data in a specific product? You can find a full breakdown of data availability by product our data availability page.
In this new version of the data pipelines, we have implemented the following improvements:
- Aligned how we process AIS and VMS data, ensuring that no matter the source, the data is handled with the same high standard of accuracy.
- Upgraded our processes and backend “engines” so we can deploy new data and features to our users much more quickly in the future.
- Removed outdated internal data tables and reorganized our datasets into clear, specialized categories.
- Released a new unified VMS pipeline that simplifies complex data workflows, allowing us to easily expand our coverage as more countries share their vessel monitoring data.
While many of these technical enhancements happen behind the scenes, they are foundational to our commitment to improve our data. These upgrades allow us to refine and improve our datasets more frequently and reliably. As a reminder, our users can always stay informed about specific known issues by visiting our Data and Platform Updates page here.
Automatic identification system
There are no changes to the data values or logic in this release. Our updates were primarily focused on backend infrastructure to improve system performance and reliability.
Known issues:
Vessel Identity
- MMSI to ISO 3 fix: We have refined the logic for determining vessel flags (Country ISO3) from MMSI numbers. Previously, we assigned flags based on the first three digits regardless of the total string length. The updated logic now requires a standard 9-digit format to validate and assign a flag. This change has adjusted the flag data for approximately 4.5 percent of unique MMSIs in our data.
- Identity data lag
- The Issue: Previously, the vessel information data experienced a data lag between January and May each year. Because the characterization model refreshes semi-annually (June 1 and Dec 1), the pipeline relied on the previous year’s data during this window. This resulted in missing new vessels and delayed identity updates, which are key for calculating AIS fishing effort, AIS events and AIS presence.
- The Fix: We have implemented a new prioritized logic to ensure improved vessel coverage:
- Primary priority: Integrate vessel information immediately as it becomes available.
- Secondary priority: If no new data exists, carry forward the most recent known values to maintain record continuity.
- User Impact
- Improved accuracy: Historical and current data for the January – May period now includes vessels that were previously omitted.
- Data availability: New vessel identities will appear in Events immediately. Updates to Fishing Effort and Presence datasets will follow with a standard one-month processing latency.
Object Detection
There are no changes to the data values or logic in this release. Our updates were primarily focused on backend infrastructure to improve system performance and reliability.
Vessel monitoring system - Country-specific data
To streamline and accelerate the integration of country-specific data, we have transitioned to a unified VMS architecture. This new framework standardizes data ingestion, significantly reducing the lead time required to launch new countries. Under the new architecture, we implemented an exhaustive data quality assurance process in version 4. This update facilitates:
- Resolution of legacy inconsistencies and automated bug fixes.
- Integration of new datasets and deprecation of other ones.
- Identification of known issue cases for future development cycles.
Resolution of issues
- Fishing effort and vessel presence
- During review of the new VMS pipeline, we identified data issues in the previous version that caused fishing effort and vessel presence to be incorrectly estimated across multiple VMS countries. These issues layered on top of one another, with some causing overestimation and others underestimation. All issues have been resolved in the version 4 release. We will soon publish a comparison of fishing effort and vessel presence hours per country.
- Ecuador time-shift issue
- Issue: When comparing AIS tracks with VMS tracks for Ecuador vessels (matched by shipname), all vessels showed a consistent 5-hour time offset. This happened because, prior to April 13, 2022, the timestamps were reported in local time, and an extra 5 hours was added to convert them to UTC.
- Fix: Starting April 13, 2022, timestamps are reported directly in UTC. In Pipe 4, the processing logic was updated to stop adding the extra 5 hours for all data on or after that date, eliminating the time offset.
- Papua New Guinea – gaps in data filled
- Throughout 2025, we troubleshooted an issue that impacted 7 vessels in PNG:
- Alpine Rose (IMO: 7523829)
- Sophia Martina (IMO: 8748696)
- Gabrielle L.T. (IMO: 8748672)
- Tobias Miguel (IMO: 8748658)
- Lauren Marie Taylor (IMO: 8748517)
- John Fisher (IMO: 8996188)
- Joe Turner (IMO: 9097355)
- For these 7 vessels, there were many gaps in data transmission from the original source iFIMS throughout 2024 and 2025 and in the past few months, we received the missing data. This data has been processed and is available in the new version of the pipeline.
- Throughout 2025, we troubleshooted an issue that impacted 7 vessels in PNG:
- Belize, Papua New Guinea, and Panama – Speed Issues
- Belize and Papua New Guinea: since 2024, we have stopped receiving speed in the reported data for Belize. For PNG, reported data largely do not include speed across the full time period. To address this, in both the previous pipeline version and version 4, when speed is unavailable, the pipeline computes speed using implied speed (derived from consecutive positions for the same country identifier). This computation requires a 1-day padding so the first position of each day can reference a position from the previous day. In the previous pipeline version, this 1-day padding was not applied, so the first position per country identifier per day often had empty speed, which contributed to an underestimation of fishing presence and fishing effort. In the new version 4, this is resolved because we apply a four-day reprocessing window, providing the prior-day context needed to compute implied speed more consistently.
- Panama raw data combines historical and daily feeds for multiple fleets (INTERNATIONAL, TRAWLER, and LONGLINE). In the previous pipeline version, we only included the INTERNATIONAL fleet, where historical speed values were kept as-is while daily speed values were divided by 10 to convert them to knots. In the new pipeline version 4, based on comparisons between reported speed and implied speed, we concluded that historical speed values across fleets are likely reported in knots and should not be scaled, while daily speed values for all fleets should be divided by 10. This remains an approximation because it relies on implied speed, so we continue working with Panama to validate historical speed units.
Integration of new datasets and deprecation of other ones
- Panama and Peru – 2012 and 2013 data added
- When reviewing what data we had available per country, we noticed that we had data for Peru & Panama for 2012 & 2013 that we had not previously processed. This data is now available in the new pipeline version 4.
- Belize Fishing Effort – Degradation in quality
- In recent years, Belize has stopped providing information about vessel type. Since this information is a minimum requirement to run the fishing model, the degradation in data quality means that fishing effort can no longer be published reliably, and we are no longer publishing it in the new version 4 of the pipeline. We are working with Belize stakeholders regarding this issue. Once the minimum data requirements are restored, fishing effort can be republished in the future.
Identification of known issue cases for future development cycles
- Belize
- Raw data counts are consistent across the old and new version of the pipeline, with the exception of 2025-10-23 and 2025-10-24, where version 4 is missing approximately 1.3 percent of data. Given the small amount of data affected, no action has been taken for now.
- Brazil
- An ~8 percent increase in country identifiers is observed in version 4 compared to the previous version for August 2025. It is currently unclear whether these vessels represent non-fishing vessels, and the team is working with Brazil to obtain more reliable vessel classification (vessel type) data to help clarify this.
- Chile
- Missing raw data: For the aquaculture fleet, approximately 3.3 percent of data is missing in version 4 on 2025-09-18 and 2025-09-19. Given the small amount of data affected, no action will be taken for now.
- Country identifier changes (caveat): Country identifiers in the previous version were generated using shipname only; version 4 uses both shipname and callsign for more accurate vessel track separation. However, because much of the 2019 historical data was delivered with empty callsigns, tracks from different vessels may be incorrectly grouped, and the transition to populated callsigns mid-2019 causes a country identifier count spike. As this affects only the 2019 historical data, no action will be taken for now.
- Shipname suffixes: Suffixes in parentheses are preserved during normalization to help distinguish vessels, with the exception of (L, (LB), and (L/B), which are grouped together. Some edge cases (e.g., TRUZAL IV vs. TRUZAL IV (BZA)) may not be fully resolved, but no further action is planned beyond the existing exception handling.
- Cross-fleet duplication: Consistent cross-fleet duplication of 7-16 percent per month from 2019–2025, with positions appearing across combinations such as aquaculture/transport, aquaculture/small_fisheries, and industry/transport.
- Ecuador
- A known caveat carried over from the previous version: from March 2021 to May 2025, positions show a consistent ~5-hour gap. Before 13 Apr 2022, the gap appears around 00:00–05:00 UTC (19:00–24:00 local time, UTC−5). From 13 Apr 2022 onward, the gap shifts to 19:00–00:00 UTC (14:00–19:00 local time, UTC−5). The root cause is not well understood.
- Panama
- Cross-fleet duplication: Cross-fleet duplication is generally very low, with the exception of a large spike in August 2024, where positions appeared across longline/trawler, international/trawler, international/longline, or all three fleets simultaneously.
- Peru
- Raw data previously missing in version 4 for the period January 2012 to July 2015 has been added. However, the data used to determine fleet category is empty for this entire period, causing all 2012–2015 records to appear with an undefined fleet category and empty shiptype. As this field is not available in the historical data and the limitation is confined to historical records only, no action will be taken for now.
- Cross-fleet duplication: Cross-fleet duplication is present but low, under 1.2 percent per month, with positions appearing across artisanal/small-scale, artisanal/industrial, and industrial/small-scale combinations.
- Papua New Guinea
- The country identifier field was updated between versions: the previous version used callsign, while version 4 uses shipname. In most cases, shipname and callsign have a one-to-one match. The only exception is vessel MARITA 88, which had two different callsigns; however, track analysis confirmed that both refer to the same vessel.
- The country identifier field was updated between versions: the previous version used callsign, while version 4 uses shipname. In most cases, shipname and callsign have a one-to-one match. The only exception is vessel MARITA 88, which had two different callsigns; however, track analysis confirmed that both refer to the same vessel.
- Other known issues
- Registry Matching: Some information like the gear type is based on countries’ registry information which in some cases does not match our VMS vessel information perfectly which requires us to fuzzy match registry data. This process can lead to incorrect matches and is currently unstable so the geartype might randomly flip from one day to the next.
- High volume of empty vessel types: The highest percentages of empty vessel types are in Belize (~92 percent), Ecuador (~12 percent), and Panama (~11 percent) of vessels. We will be working closely with country partners to improve both the availability and the definition of vessel type in the source data.
As of Feb. 26, 2026 , these changes apply to Global Fishing Watch’s:
- Map, Marine Manager and Vessel Viewer tools.
- Public APIs, gfwr (R package) and python package.
This update is still not available in the data download portal. We’ll let our users know when it is available.
If you have any questions about this release or our products, please contact [email protected]
