4  So you ran an assessment model, more notes and best practices

4.1 Retrospectives

  1. Retrospective analysis is a way to assess whether something is inconsistent in data or model assumptions. A retrospective pattern can arise from changes in catch, M, q, selectivity, or growth from that which is in the model. It is primarily an exploratory/diagnostic tool, though see point 4 below. The {nwfscDiag} package can help with this.

  2. See the 11/17/2021 PEP team meeting presentation for background information and past publications on retrospective analysis

  3. PEP’s approach is to apply:

    1. 5 years of peels (based on findings from Miller and Legault 2017),

    2. using the alternative Mohn’s rho (Mohn’s rho averaged over peels - the AFSC_Hurtado term from r4ss::SSmohnsrho),

    3. for depletion, biomass, fishing mortality, and recruitment. Often biomass, depletion, and recruitment are provided in assessment reports

  1. Although the east coast uses mohn’s rho to “correct” status indicators or when setting quotas, our practice is not to. This is based on past precedent, and that our fishing history is not as long as it is on the east coast.

  2. Hurtado-Ferro et al. (2015) provide a rule of thumb on the significance of mohn’s rho (average over peels) dependent on life history. They suggest a retrospective pattern is not meaningful if \(\rho \in (-0.15, 0.2)\) for long lived species and \(\rho \in (-0.22, 0.3)\) for short lived species, and note that magnitude of Mohn’s rho not related to true bias in assessment. Miller and Legault (2017) argue that variance of Mohn’s rho is truly needed to ascertain whether an effect is significant.

4.2 Conducting a rebuilding analysis (aka Puntalyzer)

A rebuilding plan will need to be developed for any species (or stock) that has an GFSC SSC approved stock assessment that estimates the relative spawning biomass to be below the corresponding minimum stock size threshold (0.10 for flatfish and 0.25 for all other groundfish species). A rebuilding analysis is developed using a software program called the “rebuilder” (aka the Puntalyzer) developed and maintained by Andre Punt (aepunt@uw.edu). The program is designed to work with an input rebuilding data file created by Stock Synthesis called “rebuild.dat”. The rebuilder software executable then develops numerous future projections and calculates the probability of rebuilding based on alternative harvest strategies. The RES.CSV created by the rebuilder executable contains all of the resulting estimates of rebuilding (although many quantities are not labeled within the CSV file).

The rebuilder github repository (pfmc-assessment/rebuilder) contains the most up-to-date rebuilder executable, the user manual, code tools for processing output, and code examples. This repository should have all the pertinent information to conduct a rebuilding analysis for a species or stock managed by the Pacific Fishery Management Council

4.3 r4ss

  1. The {r4ss} package should be installed from GitHub. Installation instructions are available at https://r4ss.github.io/r4ss/.

  2. See the intro vignette at on that website for info on more advanced topics like scripting Stock Synthesis workflows with r4ss and using the r4ss::tune_comps() function.

  3. It’s good idea to reinstall r4ss on a regular basis to keep up. You can also watch the github repository if you want to get notifications of changes.

  4. Look through all the figures that get produced by r4ss::SS_plots(). There have been cases where models were put forth with incorrect assumptions about biology, ageing error, etc. that could have been caught if the assessment authors had paid attention to all the plots.

  5. Remember that the figures can be modified to look better. For instance, you can replace the fleet names in the model with a less abbreviated set of names that will go in the plot labels.

  6. If something is not working right, complain about it on the r4ss issues list.

4.4 Writing assessment documents

  1. Read the terms of reference and actually follow them.

  2. As of winter 2024, we are transitioning from {sa4ss} to {asar} but don’t yet have any regional guidance to add to this handbook.

  3. Use proper dashes and hyphens for readability: Hyphen, En dash, and Em dash

  4. Jason Cope suggests comparing assessment results with estimates from simpler methods saying: “It would be really easy to compare results of every assessment we do with the results of the data-poor methods.” Jason did this for cabezon in 2019 and thinks we could do it for all our assessments.

  5. The Council website has a useful dashboard for information on recent fishery performance, OFLs, ACLs, etc. Go to the “GMT016 - Stock Summary” table and filter by species and area.