JO 7210.633A

10/01/2020 JO 7210.633A Chapter 2. Safety Data 1. Introduction. A successful safety system is highly dependent on accurate and timely data collection. AJI is responsible for collecting all safety-related data within the ATO. AJI will use collected data to properly identify and categorize reports, occurrences, and risks. Data will be analyzed for system trends and potential risks and will be primarily used to evaluate and possibly mitigate identified risk, including national orders, processes, and procedures. These data are not used as the basis for individual performance management, nor should it alone typically drive SDP quality control (QC) programs or actions. ARIA data or the suspected presence of risk in an operation may be used to initiate the CSBR process, which will generate a cooperative discussion between QA and an SDP.

2. Release of Safety Data. AJI is the sole ATO office responsible for coordinating the release of safety data to organizations outside the ATO. Other organizations must not release safety data outside the ATO without prior coordination with AJI.

3. Data Usage. AJI will collect safety data to: a. Accurately identify, categorize, and reconcile reported occurrences.

b. Accurately analyze aircraft encounters.

c. Identify suspected National Airspace System (NAS) risk trends.

d. Assess the effectiveness of NAS risk mitigation actions.

e. Analyze NAS services.

f. Analyze NAS policies and procedures for compliance and effectiveness.

4. Collection Methods. AJI will collect safety data via the following methods: a. CEDAR. AJI will analyze data provided by ATC facilities and Technical Operations districts through the MOR submission processes or collected automatically via ARIA within the CEDAR application.

b. Remote Collection. To minimize impact on ATC facilities’ and Technical Operations districts’ operations, AJI will conduct remote data collection to the greatest extent practical, using automation such as the National Offload Program, FALCON, ARIA, Performance Data Analysis and Reporting System, Digital Audio Legal Recorder, Remote Monitoring and Logging System (RMLS), etc.

c. Additional Data Requests. AJI may request additional data, such as voice and radar/replay data or NAS services data. The purpose of such requests is to accurately analyze an operation/occurrence and is not intended to initiate a review of the operation/occurrence by the facility. AJI will indicate the format in which the information should be prepared, the scope of the information requested, and the method or means by which the information will be delivered.

All data requests must be fulfilled within two administrative days of the request being received.

2-1 10/01/2020 JO 7210.633A d. ATC Facility and Technical Operations Services District Support of Analysis Data.

The facility must ensure that all pertinent data are compiled and forwarded to AJI in support of the QA safety analysis process. Upon notification by AJI that additional data is required to assess the operation/occurrence, the facility must ensure the following: (1) If requested by AJI, all pertinent voice files must be attached to their associated reports in CEDAR within two administrative days. At a minimum, these voice files must include all communications with and about the aircraft involved in the reported operation/occurrence from either initial contact with the operating position responsible for the aircraft at the time of the operation/occurrence or five minutes before the operation/occurrence, whichever happens last. The voice files must continue until five minutes after the operation/occurrence or until final contact with the operating position, whichever happens first.

(2) Include any recorded telephone conversations with involved flight crews concerning the operation/occurrence. Ensure that operation/occurrence-related communications with flight crews are conducted over recorded lines wherever available.

(3) Although not required, every effort should be made to provide a bookmark with audio if the capability exists, in lieu of just providing a requested voice file. If FALCON is unavailable, or audio only is provided, attach audio data files that are .wma, .avi, or .mp3 format, due to file size and bandwidth restrictions.

NOTE – Once notified by AJI that an operation/occurrence has been classified as a pilot deviation or is supporting other litigation, facility management must ensure that the voice recordings or acceptable waveform audio file (.wav) is retained according to applicable orders (see Chapter 2, Paragraph 6, Documentation Retention).

NOTE – Technical Operations Services uses databases other than CEDAR. Until CEDAR is fully implemented throughout the ATO, AJI data requests will require that Technical Operations Services personnel pull data from existing databases.

5. Information Requests. Information requests provide data needed to make informed decisions. They are for gathering additional data and do not require corrective action. Requests to facilities or other ATO organizations may be either informal or formal, but formal processes are documented and tracked. Formal information requests may be generated in those situations where informal data gathering may be impractical, not expedient, or otherwise inappropriate. As a result of receiving a request, recipients may choose to request a CSBR and/or initiate QC processes, which could result in corrective action.

a. AJI initiates formal information requests. The request initiates a bottom-up information exchange directed at the level most closely related to the issue.

(1) Recipients must provide a response within the identified timeline.

(2) If recipients do not provide a timely response, the issue may be escalated to a CAR.

b. If the response confirms the identified safety issue, AJI will work with the respondent for resolution. If further action is required, the issue may be elevated to a CAR.

2-2 10/01/2020 JO 7210.633A 6. Documentation Retention. FAA Order JO 1350.14 provides general requirements for data and record retention. FAA Order JO 8020.16 provides retention requirements for aircraft accidents, aircraft incidents, litigation, and enforcement support.

The following are the retention requirements for reports and occurrences covered by FAA Order JO 7210.632 and this directive: AJI must retain all data collected through the MOR and ARIA processes, in accordance with retention policies.

2-3

10/01/2020 JO 7210.633A Chapter 3. Analysis 1. AJI Responsibilities.

a. Providing risk-based trend information, statistical data, recommendations, and other pertinent information to assist field facilities and other ATO organizations with their mitigation efforts. Information and data should be used in conjunction with field facility QC data to understand and mitigate areas of risk.

b. Analyzing safety data from NAS Services performance data (e.g., RMLS, National Airspace Performance Reporting System).

c. Examining and reconciling reports collected through automated and MOR processes to ensure: (1) The quality of the data is of the highest standard.

(2) Accurate categorization of operations/occurrences.

(3) Identification of air traffic incidents that must be reported to other organizations (e.g.

pilot deviations, surface events, etc.).

d. Conducting risk-based analysis of air traffic operations in accordance with this order and supporting AJI standard operating procedures. Ensuring that findings are made available to the ATO (e.g., observed and identified national trends, recommended mitigations).

2. Identification and Calculation. AJI will identify or calculate the following from reconciled data: a. Potential systemic risk.

b. Applicable ATO and agency safety metrics.

c. All runway incursions/excursions.

d. Pilot deviations and near-midair collision reports, which AJI will forward to the responsible Flight Standards office.

e. Vehicle and pedestrian deviations, which AJI will forward to the Airports Division and other affected organizations.

f. Spillouts, military deviations, etc., which AJI will forward to the Department of Defense.

3. Quality Assurance Validation.

3-1 10/01/2020 JO 7210.633A a. ARIA Validation. QA will analyze operations and occurrences to support the identification of potential risk in the NAS, regardless of compliance with separation standards.

PARs are captured and reviewed: (1) Automatically, based upon a predetermined set of data points contained in the QA Standard Operating Procedures.

(2) Manually, as the result of additional referral methods into the BAR process (see Chapter 3, paragraph 4.b.(1)(a), Additional Referral Methods for BAR).

NOTE – Data that does not meet the requirements to become a PAR will be retained for possible analysis.

b. MOR Validation. QA validates MORs. Data from validated occurrences will be reviewed for potential systemic risk and are not a trigger to initiate performance management.

4. ARIA and Barrier Analysis.

a. Administration of ARIA. AJI is responsible for the administration of ARIA. QA is responsible for identifying and validating risk in the NAS. ARIA is a tool that facilitates this activity at a systemic level. The ARIA software platform and associated algorithms are designed to make an initial finding of potential risk in the NAS. When an initial finding meets a specific set of data points, it is referred to BAR. BAR is conducted by subject matter experts, where measurements of risk associated with individual operations are quantified and associated factors are assigned. Aggregate BAR data is then used to accurately identify the presence of risk on a systemic level, determine mitigation efforts, and monitor their effectiveness. ARIA and the associated QA and QC processes protect the confidentiality of individuals involved in affected operations.

b. Barrier Analysis Processes. Barrier analysis processes are the means by which ARIA data is evaluated to determine risk.

(1) Barrier Analysis Review. BAR is the assessment of safety data to determine systemic risk. Additionally, the review captures aggravating, mitigating, and observed factors, and includes an evaluation of the resiliency of available safety barriers. The aggregate data produced by this process assists in the identification of systemic trends and potential risk in the NAS.

(a) Additional Referral Methods for BAR. In addition to RARs, the following methods are included in BAR: i. Potential operational risk identified during the QA validation process.

ii. Service Area QA Manager referral.

iii. NATCA Service Area Safety Representative referral.

iv. AJI Headquarters Manager referral.

3-2 10/01/2020 JO 7210.633A v. NATCA National Safety Representative referral.

vi. Random selection by QA.

(b) BAR Referral Requests. The following BAR referral requests are subject to approval through a collaborative review within the applicable Service Area QA office: i. AJT Management referral request.

ii. Local Safety Council (LSC) referral request.

(2) CSBR. A CSBR utilizes aggregate data from the BAR process, and other operational data, that may indicate potential risk in the NAS at the SDP level or above. The review will be conducted cooperatively between the QA office and the facility or facilities. The review utilizes multiple data sources to analyze and identify systemic risk. The identification of systemic risk within the NAS allows the ATO to mitigate and track operations that might pose a risk.

(3) CSBR Referral. Participation in the CSBR process initiated by QA is mandatory.

CSBR panels are established: (a) If more information is required to fully understand an operation being reviewed within the BAR process.

(b) If potential systemic risk has been identified from the BAR process, and mitigation is believed to be warranted.

(c) As the result of a safety concern identified during an on-site event review by an Event Response Team.

REFERENCE − FAA Order JO 1030.3, Chapter 5, On-Site Event Review.

(4) Requesting CSBR. The following requests are subject to approval through a collaborative review within the applicable Service Area QA office: (a) Facility request.

(b) District request.

(c) LSC request.

(5) Documenting CSBR. The review process must be documented in CEDAR in the Combined Safety Barrier Review Tool (CSBRT). All areas, as noted in the CSBRT question tree, must then be considered, and related documentation and data used to support the outcome of the review must be referenced or attached in CEDAR in the tool. Data to consider should include: (a) Associated previous BAR information and results, to include aggregate data from previous RARs.

3-3 10/01/2020 JO 7210.633A (b) SDP QC data, to include: i. Service Review data.

ii. QC Operational Safety Assessment data.

iii. MOR data.

iv. Current and previous Corrective Action Plan (CAP) information.

v. Compliance Verification data.

vi. Partnership for Safety, Safety Data Portal (facility must have an LSC).

vii. Any other data deemed relevant by the collaborative team.

(6) CSBR Resolution. Once an issue is understood (assessed and analyzed), facilities must collaboratively develop a CAP in accordance with FAA Order JO 7210.634 and the NATCA/FAA Collective Bargaining Agreement, or document why mitigation is not required, and close the CSBRT in CEDAR.

3-4

10/01/2020 JO 7210.633A Chapter 4. Communications 1. Recurring Reports. AJI will provide NAS safety data and trend findings as required by the Chief Operating Officer and other senior executives. These reports will also be provided to external organizations and labor unions, as appropriate.

2. Corrective Action Requests. CARs are formal requests initiating action to resolve an identified concern. The CAR process is initiated when it becomes unlikely that identified safety issues will (or should) be resolved informally. CARs begin a top-down process to inform the ATO of reported safety issues.

a. CARs typically identify systemic safety issues and are rarely based on a single data point.

b. All available information must accompany the request.

c. Recipients must provide a response within the identified timeline. Although some issues are complex and require additional time to develop a comprehensive CAP, a response indicating the ATO’s intentions is still required within the identified timeline.

d. If recipients do not provide a timely response, or if the recipient requests an extension, AJI will notify the appropriate Vice President.

e. If AJI does not initially concur with the response, AJI will work with the respondents in order to achieve resolution.

f. If the corrective action plan is sufficient, the issue will be closed.

g. If AJI determines that the CAP does not mitigate the issue, AJI may elevate the matter to the ATO Deputy Chief Operating Officer to pursue appropriate action.

4-1