1 Data sources: notes, contact information, and links
1.1 Table of contacts
Contact info for various agencies (states, PacFIN, etc.) is listed in this Google Doc created by Jason Cope in 2009 (updated May 2021)
1.2 Requesting data from State partners
Starting with the 2023 cycle, there is a more formal process for making data requests to states for stock assessments. Details about the data requesting process, and steps for assessment leads to take, can be found in the document “Process for requesting data from states for 2023 cycle.”
1.3 Regulations and catch limits
Jim Hastie has lived through many changes in regulations for west coast groundfish. Don Pearson has recently (Fall of 2014) created this online regulations database.
1.4 Catch data, lengths, ages, etc.
1.4.1 West Coast Groundfish Observer Program (WCGOP)
See section below on “Notes and best practices for observer data and discards” ADD LINK
1.4.2 At-Sea Hake Observer Program (ASHOP)
Vanessa Tuttle (Vanessa.Tuttle@noaa.gov) is the contact person for this. Unlike the shoreside subset of the hake fishery which has dockside sampling and bycatch reported in PacFIN, the observed bycatch in the at-sea sectors (motherships and catcher-processors) is reported to the NORPAC database housed at AFSC. A non-definitive list of “common” and “uncommon” species seen in the hake fishery (as of 2008, but Vanessa says it is still current as of 2022) is in Appendix A of this document.
1.4.3 PacFIN biological data (BDS)
To get these data, create a “Data request” issue on the PacFIN.Utilities github repo. To confirm sample sizes for BDS, go to this webpage which should give a recent tally of samples by state agency.
Generalized code to process the data are currently in {PacFIN.Utilities}.
You may see some discrepancies in CA data from 1985-1989 when compared to older extractions. Some samples were removed for some species. It is not clear why these samples were removed.
1.4.4 PacFIN landings
To get these data, create a “Data request” issue on the PacFIN.Utilities github repo.
Note that the PacFIN catch can be broken down by INPFC or month only for the trawl component, not the non-trawl gears. Also, beware double counting based on multiple levels of aggregation.
PSMFC areas do not contain all of the catch, thus it is best to use INPFC areas to aggregate catch.
Once you have catch values for all years, contact the state representative(s) for confirmation that the values are correct.
1.4.5 RecFIN
All states have in the past claimed that all rec. data is available on RecFIN. Users should check with state representatives for updated information on the recommended source for recreational data. One can pull historical catches here. Select the “Catch / Sample Data Reports” image which takes you to the reports dashboard. Alternatively, ODBC connections can be used to pull catch estimates directly from the Comprehensive Fish Ticket table. Some data sources (e.g. MRFSS, WA historic catch estimates) are not currently available via the public login, so you need to login with an account that has permission to view the tables. Additionally, once you have catch values for all years, contact the state representative for confirmation that the values are correct.
Washington provides historical recreational catch in numbers. (Modern catches are available in both numbers and weight.) This leads to two decisions for putting catches into the model: keep catch in numbers, which stock synthesis converts to weights internally; or convert catch to weight using an external estimate. Both options have been done historically. The former causes challenges for projections, but if the former is chosen, it is useful to compare the model’s calculated average weight of the catch to the value used to convert numbers to weight to see if there is large disagreement. For a partial history about this issue, and approaches for some past assessments see issue #52 from the canary rockfish github page. Note that Washington historical catches are publicly available on RecFIN, but should be filtered to areas 1-4 (coastal marine catch areas). As of the 2025 yellowtail assessment, publicly available RecFIN catch data excluded all Puget Sound catch, whereas catch from the “Sekiu and Pillar Point” area near the entrance to Puget Sound should be included in assessments in federal waters. In order to include these catches, you will need to use the confidential catch data table (CTE501), rather than the public version (CTE001), which aggregates data to a higher spatial resolution and excludes Puget Sound.
To obtain length composition data,
- Go to RecFIN
- Select the “Catch / Sample Data Reports” image which takes you to the reports dashboard. There, select the “SD001 Biological Detail Report” option.
- There is an automatic filtering applied, so to adjust select the ‘filter’ icon in the upper right (the upside down Erlenmeyer flask-like icon), and then download your data in either csv or excel format. Lengths come in imputed and measured, with T being total length and F being fork length. For questions on definitions of fields, the metadata is included as a selectable report in the reports dashboard.
E.J. Dick (SWFSC) and Jason Edwards (PSMFC) are developing tables with catch-weighted length compositions. Tables have been prepared for WA and OR, and CA is in prep. Contact E.J. (edward.dick@noaa.gov) if you have questions or would like to use the estimates.
1.4.6 Research catch
This is not automatically included in any of the other data sources. Gretchen Hanshew (Gretchen.Hanshew@noaa.gov) has been the source in the past. Talk to John Wallace and Ian Taylor about complexities regarding PacFIN records of landings from research catch.
1.5 Indices of abundance
At a team meeting on 13 March 2019, the team agreed on the following terminology for the surveys, where best practice would be to introduce the survey initially using the full name, with the short name in parentheses, and then use the short name after that. This will be the approach used in the write-up of the surveys as well.
NWFSC West Coast Groundfish Bottom Trawl Survey (WCGBT Survey)
AFSC/NWFSC West Coast Triennial Shelf Survey (Triennial Survey)
AFSC West Coast Slope Survey (AFSC Slope Survey)
NWFSC West Coast Slope Survey (NWFSC Slope Survey)
NWFSC Southern California Shelf Rockfish Hook and Line Survey (H&L Survey)
NWFSC Integrated Acoustic and Trawl Survey of Pacific Hake (Acoustic Survey)
Note: the names for the first 4 of these surveys in the {nwfscSurvey} package are “NWFSC.Combo”, “Triennial”, “AFSC.Slope”, “NWFSC.Slope”. The H&L and acoustic surveys are not currently available through that package.
1.6 Summary of NOAA fishery independent trawl surveys used for west coast assessments
Triennial Survey (1980–1992 & 1995–2004)
early triennial (1980–1992, 55-366m, north of 36.8 (however the switch to north of 34.5 actually occurred in 1989))
late triennial (1995–2004, 55-500m, north of 34.5)
The 2007 Canary assessment (Stewart, 2007) seems to have been the first one to split the survey and includes info on justification (but there’s an ongoing debate about the need to split).
Must filter out water hauls and tows occurring outside the US EEZ (foreign tows)
Note that 1977 is always tossed out.
The 2005 and 2013 Shortspine Thornyhead assessments split the triennial into shallow vs. deep to create a single index from 1980-2004 for 55-366 m and a separate 366-500 m index for 1995-2004.
AKFSC Slope Survey (1997-2001, 183–1280m, north of 34.5)
Years before 1997 surveyed small areas of the coast
Must filter out tows occurring outside the US EEZ
NWFSC Slope Survey (1998-2002, 183-1280m, north of 34.5)
WCGBT Survey (2003-present, 55-1280m, entire US coast)
Starting in 2004, there’s a change in sampling intensity north and south of 34.5, so this strata boundary should be included (unless there’s some specific reason not to)
Note that there were changes in sampling intensity at 183 and 549 meters
This survey should be referenced as the “WCGBT Survey”
1.6.1 AFSC surveys
Data are now available through the NWFSC data warehouse which can be accessed by the functions in the {nwfscSurvey} package. Appendix B identifies which year the listed vessels participated in the Triennial and Slope Surveys. Each cruise is assigned a unique number which is contained within the ‘CRUISEJOINS’ column in the database. Checking that your data has all the cruises you expect for a given survey would be good practice. Although gear and personnel change over time, if a study looking at differences by vessel was undertaken, Appendix B shows which vessels participated in what year and survey.
1.6.1.1 More detail on these AFSC surveys
The the most recent RACE division species and data codes manuals are here.
The ADP Code Book has, for example, sex and performance code information. For convenience, here is the legacy coding for sex:
Sex
1 Male
2 Female
3 Undetermined
Note on design of the 2004 Triennial Survey
It is my (John Wallace) understanding that in the later years of Triennial survey (pre-2004), the survey became more of a fixed survey design as the skippers went back to the same locations as recorded on their vessel’s instrumentation.
For the 2004 survey, I followed the design as put forth in:
2001 AFSC Triennial Survey Plan (converted from WordPerfect via Word)
without regard to any previously recorded tow locations.
For CRUISEJOIN info, see Appendix B.
1.6.2 NWFSC Survey Indices
Tech memo on “history, design, and description” of the survey is now available:
https://www.nwfsc.noaa.gov/assets/25/8655_02272017_093722_TechMemo136.pdf
Keller, A. A., J. R. Wallace, and R. D. Methot. 2017. The Northwest Fisheries Science Center’s West Coast Groundfish Bottom Trawl Survey: History, Design, and Description. U.S. Department of Commerce, NOAA Technical Memorandum NMFS-NWFSC-136. DOI: 10.7289/V5/TM-NWFSC-136.
Additional information on the survey can be found in these documents from John Wallace:
Strata Tow Percentages for NWFSC Bottom Trawl Survey for 2004-Current
Index standardization is used Kelli Johnson’s VAST wrapper has specific examples for each survey located in the inst\examples folder.
1.6.3 NWFSC Survey Length and Age Compositions
{nwfscSurvey} package is used for comp data and other data explorations.
Scaling from Tow to Stratum Level: Weight normalized length or age comps for each tow by the numerical CPUE. This is done in the standard data package we get from Beth.
Scaling from Stratum to Coastwide (or Assessment Area) level: Weight strata length or age comps by numerical index for each strata (from GLMM). This may mean dividing the biomass index for each stratum by the average weight in that stratum (likely estimated from the length comp in that stratum).
1.6.3.1 Filtering recorded catches that are fish stuck in net from previous tow
In some cases, fish caught in one tow remain in the net until the next tow and are recorded as caught in that second tow (despite the attempts of the people on the survey to identify and exclude such fish from the data). While we plan to come up with a consistent way to deal with this, it has not been dealt with yet.
One problem arising from this is catch data outside of the depth range of the species. To identify the depth range, one can start with the deepest/shallowest tow with a positive record for that species and then look to see if the previous tow conducted by that vessel (likely on the same date) caught that species. If so, and if a small number or a small relative number of that fish species were recorded in the second tow, one can assume that those were from the previous tow. By moving sequentially until one reaches a depth range where a few clearly legitimate tows occurred, one can define a reasonable depth range.
A second phase would be to define a way to filter all the tows to remove likely candidates. A simple way to do this is to look at the range of catch levels that were determined to be not legitimate, and filter the whole data set for that species by removing all tows with catches below some quantile of those catch levels (say 80 or 90%). This will eliminate having a large number of small catches in the database used for the GLMM, which would skew the modeled distribution.
1.6.4 IPHC survey
This survey has been used for Yelloweye Rockfish and Spiny Dogfish. Claude Dykstra (claude@iphc.int) has provided data in the past. The index was calculated using a binomial GLM developed by John Wallace. Talk to John or Ian Taylor to get R code for this analysis.
1.6.5 Oregon recreational observer program data
Troy Buell (troy.v.buell@state.or.us) is still involved with this data source even though he has changed positions.
1.7 Notes and best practices for observer data and discards
Discard mortality by fleet should be accounted for within the assessment. To obtain estimates of total discards based on the fleet structure in the model (gear and/or area) see the Groundfish Expanded Multi-year Mortality (GEMM) report which is a component of the annual Groundfish Mortality Report. The GEMM provides annual estimates of landed and discarded fish by sector off the West Coast starting in 2002 - present. However, the GEMM will not include the most full recent year estimates (example - the GEMM available in 2019 has data through 2017) which will require an assumption regarding the most recent year’s discards. Additionally, assumptions will need to be made regarding the total amount of discard prior to the start of the WCGOP data (pre-2002).
The GEMM report is produced by the WGCOP team with Dr. Kayleigh Somers (kayleigh.somers@noaa.gov) as the lead author and is published in the September Council meeting Briefing Book titled the “Groundfish Mortality Report”. The GEMM can be found within the associated Excel spreadsheet on the Table 3 tab. Additionally, the GEMM can be accessed by using the nwfscSurvey using the pull_gemm() function.
There are typically three common methods used within West Coast groundfish stock assessments to incorporate discard mortality.
The first approach is to include discard mortality into the catch data rather than estimating discards within the model. Discard mortality can be included by either adding this total to the catch by fleet or by adding a discard fleet. If using the discard fleet the selectivity of this fleet will need to be mirrored (see the 2017 California Blue/Deacon assessment for an example of this approach).
The second and third alternative approaches to including discard data is to model the process of discarding within SS3. Discards can be modeled either as total discards in mt or through the rate of discarding. Both of these approaches will require discard length data to be included in the model, or if not available, a specific assumption regarding the retention curve relative to the estimated selectivity curve.
Estimates of total discards are available in the annual groundfish mortality report (and the GEMM). If a specific breakdown of discard by fleet is required (gear and/or area) speak with Dr. Chantel Wetzel. The data may not be able to be summarized to all fleet structures, so it is best to discuss the options with Chantel early in the assessment process. Assumptions regarding the discard total in the most recent year and historical years prior to the start of WCGOP data (2002) will need to be made by the assessor and modeled (often through the use of time blocks). If total discards are modeled, one will need to enter an annual CV into SS3 which is currently not available with the GEMM. The current approach for obtaining discard totals or discard rates with CV is via bootstrapping the available WCGOP observer data. The bootstrap CV will be based on the observer data and may not be entirely consistent with the data available in the groundfish mortality report (and the GEMM), particularly for sectors with limited observer coverage. Please see below for additional information regarding bootstrapping.
Modeling discard rates can be an easier approach since it does not require knowledge about the total discard amounts. However, this approach may not be feasible for stocks with limited observations in the WCGOP data due to high variability in the observed discarded and retained fish. The IFQ trawl fleet currently has near 100% observer coverage starting in 2011, while the non-catch share sectors have a much lower observed percentage (~20% as of last inquiry, but this may change in the future). Historically, it was thought that discard rates could not be calculated for complex managed species. However, this is not the case based on the current data. Discard rates are calculated based on the observed discarded weight relative to the total observed weight of discarded and retained fish. Bootstrapping is done using the data in order to provide a CV to the discard rate. Similar to two previous approaches, assumptions regarding the discard rate in the most recent year and historical years prior to the start of WCGOP data (2002) will need to be made by the assessor and modeled (often through the use of time blocks).
Note, if discard mortality is estimated within the model, it is important to compare the catches (landings + discards) estimated within SS3 to the total mortality estimates from the GEMM. The GEMM is the official mortality report for West Coast groundfish stocks as is used to determine the ACL and OFL attainment and if overfishing is occurring. The total mortality estimates between SS3 and the GEMM would not be expected to match exactly, however, the should have a reasonable level of similarity. If there are large differences between the mortality estimates across multiple years one may need to reconsider if the model has enough data to appropriately estimate retention reliably for each fleet.
The SSC has specified that the GEMM is the official data source to update recent removals when conducting a catch-only update model. The GMT is currently tasked with allocating removals by modeled fleet using the GEMM. The GMT will provide the removal estimates for year with GEMM data, provide expected removals for the final model year, and removal assumptions to use during the projection period. These data will typically be available between March and April of the assessment year.
Finally, the GEMM includes recreational mortality by state. These values are not estimated by WCGOP but rather are pulled directly from RecFIN since WCGOP does not observe recreational fisheries.
1.7.1 What are the WCGOP data?
The WCGOP database includes only data collected by observers on West Coast vessels. The data available in this database varies fairly dramatically based on the sector. The IFQ vessels essentially have 100% observer coverage, so the database includes a comprehensive view of discards and retained catch from observers for that sector. However, other sectors have a much lower percentage of observer coverage (see the observer coverage). As an example, the nearshore fishery generally has about 20% observer coverage. Only the observations made on that percentage of vessels are going to be available in the WCGOP database. Hence, the observed retained and discard amounts are going to be a significant underestimate of total mortality in that sector. The discard ratio will also be more variable, but could generally be representative of the discarding behavior.
1.7.2 Available gear types
WCGOP observers a wide-range of fishing sectors for vessels participating in the Individual Fishing Quota (IFQ, also referred to as catch share) fishery and non-catch share fisheries. Groundfish stock assessment most often define fleet structures within assessment models based on gear types. WCGOP data contains data for the following gear groups: bottom trawl, fixed gears, hook & line, midwater trawl, pot, and shrimp trawl. Grouping data based upon gear types will include data from multiple sectors which may or may not catch particular species. Each gear type includes the data from the following sectors:
bottom trawl: catch shares, limited entry Pacific halibut, limited entry trawl, open access California halibut, and sea cucumber
fixed gear: nearshore
hook & line: catch shares, directed Pacific halibut, limited entry fixed gear DTL, limited entry sablefish, and open access fixed gear
midwater trawl: catch shares, limited entry trawl, midwater hake, and midwater rockfish
pot: catch shares, limited entry sablefish, and open access fixed gear
shrimp trawl: pink shrimp and ridgeback prawn.
The groundfish expanded multi-year mortality (GEMM) report can be used to understand how much mortality (landed and discarded) is coming from each sector for any particular species. Examining these data can help analysts understand how best to use these data within their assessment. The GEMM data can be accessed using the pull_gemm() in the nwfscSurvey package.
As noted above, the observer coverage varies by sector with the catch share fishery have 100% observer coverage starting in 2011. Given that, these data are treated as a census with estimates of discarding rates provided separately between catch share and non-catch share data. Additionally, electronic monitoring became available for catch share vessels starting in 2015. Vessels using electronic monitoring have video coverage of on the water discarding rather than always having an human onboard observer. These data are also considered to be a full representation of discarding.
The catch share fishery includes the following sectors: catch share, catch share electronic monitoring, limited entry California halibut, midwater hake, midwater hake electronic monitoring, midwater rockfish, and midwater rockfish electronic monitoring.
1.7.3 Bootstrapping
The current approach to obtain uncertainty around the data in the WCGOP database, either the total discard or discard rates, is to bootstrap the data. These data are summarized and the observations bootstrapped to obtain uncertainty estimated based on gear and area stratification requested by the assessor. Chantel Wetzel (chantel.wetzel@noaa.gov) currently conducts the bootstrap analysis.
1.7.4 Biological data
WGCOP contains data on the lengths of fish discarded by gear type and sector as well as the average body weight of discarded fish that can be used within an assessment to estimate discarding within the model.
Generally, retained length or margninal age composition data within assessments from the commercial fishery (PacFIN bds) or NWFSC surveys are expanded up to the tow/trip and up to the total catch weight by area. In contrast, WCGOP length or age data are only expanded up to the tow level. This is because the total discard or landed weight by area and gear is not readily available in the WCGOP data. Hence, analysts should be mindful about the observations by gear and sector to ensure that the overall compositions are representative of landed catch in the model by fleet. For example, if 50% of the discard lengths are arising from a sector with 5% of the overall landings in the model fleet, this could lead to a situation where the estimated retention curve that is not representative of the sector with the 95% of the landings in the model. Another example is the shrimp trawl fleet that generally has low overall bycatch of rockfish, but can have large bycatch numbers of small juvenile rockfish. Grouping these data together with other trawl gears will influence both the retention curve but potentially also the estimates of annual recruitment deviations. To deal with this one could choose to not group shrimp trawl length data within the trawl gear data or the analyst could request the length data separated by gear type and then weight the data by total discards available in the GEMM (e.g., one could also talk to Kayleigh Sommers if the correct stratification is not available in the GEMM).
1.7.5 Requesting discard data
There are two types of discard data available using WCGOP data. The first is a summary of the observed discards, discard rates, and bootstrapped uncertainty across years (2002 - present). The second type of data are lengths of discarded fish observed by WCGOP. To request these data please create an issue on the nwfscDiscard github repo.
1.7.6 Discard rates, and length comps, from the Pikitch et al. Discard Study (1985-87) and Mesh Study (1988-90) databases
If enough data exists, discard rates and length comps from the Pikitch et al. Discard Study database (or Mesh Study if no data exists in the Discard Study) may be obtained for a species to be assessed from John Wallace. Optionally, these rates can be expanded out to the surrounding years based on applying the study rates to PacFIN catch from the expanded years. This, of course, is only reasonable when the assumption of no significant changes to the fishery is a good one. For those in NOAA a draft of the methods paper can be downloaded here (for non-NOAA folks that are interested, contact the author at john.wallace@noaa.gov).