ISO9001 System Document Package (24) | Data Analysis and Evaluation Procedure (9.1.3)

Document Description: This procedure corresponds to clause 9.1.3 "Data Analysis and Evaluation" of the ISO 9001:2015 standard, serving as a pivotal procedure in the "Performance Evaluation" section of the Quality Management System (QMS). Its role is to systematically collect, scientifically analyze, and objectively evaluate various data generated during the operation of the QMS, converting scattered data from inspection records, complaint logs, process monitoring, supplier evaluations, and internal audit reports into management information that can be used to judge system performance and trends. By clarifying "what data to collect, who will analyze it, what methods to use, what conclusions to draw, and where to apply the conclusions," this procedure ensures that data truly supports improvement decisions. Clause 9.1.3 is often reviewed in conjunction with clauses 9.1.2, 9.2, and 9.3 during certification audits, with auditors frequently verifying whether data has been analyzed, whether the analysis results have been included in management reviews, and whether improvement actions have been generated. This procedure is applicable to all manufacturing and service enterprises, especially small and medium-sized enterprises (SMEs) that have accumulated a certain amount of inspection and operational data but whose analysis is limited to "statistical pass rates." It can be directly applied and scaled according to the size of the enterprise.

1. Purpose

To standardize the scope, responsibilities, methods, and processes for data analysis and evaluation within the company, systematically collect, scientifically analyze, and objectively evaluate various data generated during the operation of the QMS. This ensures the conformity and effectiveness of product quality, process capability, and system performance, identifies quality trends, potential risks, and improvement opportunities, and provides data support for corrective actions, management reviews, and strategic decisions, thereby continuously enhancing the effectiveness of the QMS.

2. Scope of Application

This procedure applies to the collection, analysis, evaluation, and application of all data related to the operation of the company's QMS, primarily including the following four categories of data:

1. Customer satisfaction and feedback data (results of satisfaction surveys, complaint and return data, customer churn rates);
2. Product and service conformity data (incoming inspection, process inspection, final inspection data, nonconforming product data, delivery quality data);
3. Process and system performance data (achievement of quality targets, production plan completion rates, equipment downtime, on-time delivery rates, first-time pass rates);
4. Stakeholder performance data (supplier delivery and quality performance, results of external audits, market environment changes).

The reference and analysis of external data sources (changes in laws and regulations, industry benchmarking data, market information) also fall under this procedure.

3. Responsibilities

4. Work Procedures

4.1 Data Sources and Classification

4.1.1 The company classifies data into six categories based on its nature, and each department collects the original data using the record carriers specified in the "Quality Record List":

• Customer Satisfaction Data: Satisfaction survey scores, complaint counts and classifications, return/exchange quantities, order churn and repeat purchase rates.
• Product Quality Data: Incoming inspection pass rates, process inspection pass rates (first-time pass rates), final inspection pass rates, factory sampling inspection data, customer inspection results.
• Process Performance Data: Achievement rates of quality targets, production plan completion rates, equipment downtime, on-time delivery rates, first-time pass rates.
• System Operation Data: Number and distribution of internal audit nonconformities, on-time closure rates of corrective actions, implementation rates of management review resolutions, findings from external audits.
• Supplier Performance Data: Incoming batch pass rates, supplier delivery timeliness, frequency of supplier quality incidents, annual supplier evaluation scores.
• External Environment Data: Changes in applicable laws and regulations, industry quality benchmarking, changes in customer demographics and market conditions.

4.1.2 Each department must ensure the authenticity, completeness, and traceability of the original data, following the "Record Control Procedure" for data entry. It is prohibited to fabricate or alter data after the fact.

4.2 Data Collection and Aggregation

4.2.1 The Quality Department prepares the "Annual Data Analysis Plan" by January each year, specifying the responsible departments, reporting frequencies, analysis methods, and purposes for each type of data. This plan is implemented after approval by the Management Representative.

4.2.2 The reporting frequencies for data are as follows:

• Product Quality Data: Monthly aggregated data for the previous month must be reported by the 5th of each month.
• Customer Satisfaction Data: Data from satisfaction surveys must be reported within 10 working days after the survey ends; complaint and return data must be reported monthly.
• Process Performance Data and Supplier Performance Data: Data must be reported by the 5th of each month.
• System Operation Data: Data from internal audits, management reviews, and external audits must be reported within 5 working days after the respective activities.
• External Environment Information: Data must be reported immediately upon identifying any changes.

4.2.3 The Quality Department verifies the completeness of the data submitted by each department. If data is missing or clearly abnormal, it is returned for supplementation and recorded. Monthly aggregated data is entered into the "Quality Data Aggregation Ledger" to serve as the basis for analysis.

4.3 Data Analysis Methods

4.3.1 The company selects appropriate statistical techniques based on the data type and analysis purpose. Common methods and their applicable scenarios are listed in the table below:

4.3.2 Principles for selecting statistical tools:

• Routine monthly analysis primarily uses descriptive statistics and trend charts to calculate the achievement rates, year-over-year, and month-over-month changes of various indicators.
• For recurring or batch quality issues, a Pareto Chart is used to identify the main causes, and a Cause and Effect Diagram is used to analyze the root causes.
• For key processes (such as injection molding, welding, electroplating, or customer-focused processes), a Control Chart is used to monitor process capability, calculating Cp/Cpk values.
• Analysis personnel must be trained in the relevant statistical techniques. If improper tool usage leads to incorrect conclusions, the Quality Department will organize retraining.

4.3.3 Examples of calculation methods:

• Incoming Batch Pass Rate = (Number of合格 batches ÷ Total number of incoming batches) × 100%;
• First-Time Pass Rate = (Number of batches/pieces passing first-time inspection ÷ Total number of batches/pieces to be inspected) × 100%;
• On-Time Delivery Rate = (Number of orders delivered on time ÷ Total number of orders to be delivered) × 100%;
• Complaint Rate = (Number of valid complaints in the current period ÷ Number of shipments or sales in the current period) normalized calculation;
• Process Capability Index: Cpk ≥ 1.33 indicates sufficient capability; 1.00 ≤ Cpk < 1.33 indicates capability that needs continuous monitoring; Cpk < 1.00 indicates insufficient capability requiring improvement.

4.4 Data Evaluation and Trend Determination

4.4.1 The Quality Department evaluates the analysis results against the following benchmarks:

• Target Benchmark: Compare with annual quality targets and decomposed indicators to determine whether they have been met.
• Historical Benchmark: Compare with data from the same period of the previous year and the previous month to determine trends.
• Capability Benchmark: Compare the process capability index with industry standard values (Cpk ≥ 1.33).
• External Benchmark: Compare with customer requirements and industry benchmarks when applicable.

4.4.2 Criteria for trend determination:

• If an indicator consistently shifts in an unfavorable direction for 3 consecutive months (e.g., pass rates continuously decline, complaint rates continuously rise), it is determined as a "negative trend," triggering root cause analysis and corrective actions.
• If a single indicator fails to meet the target for 2 consecutive months, it is determined as a "target deviation," and the responsible department must submit improvement actions.
• If an indicator meets the target but shows a downward trend for 3 consecutive months, it is determined as a "potential risk" and included in monitoring and preventive measures.
• If a control chart shows 7 consecutive points on the same side, a trend of rising or falling, or other out-of-control rules, the process is determined to be out of control and handled according to the "Nonconforming Product Control Procedure" and "Nonconformity and Corrective Action Procedure."

4.4.3 Evaluation conclusions are categorized into four types: Good (meets targets and shows a positive trend), Normal (meets targets and shows a stable trend), Concern (meets targets but shows a downward trend or is close to the critical point), and Abnormal (fails to meet targets or the process is out of control). The corresponding actions are detailed in section 4.5.

4.5 Application of Analysis Results

4.5.1 Analysis results are applied through the following channels:

• Improvement Decisions: For "Abnormal" conclusions, the responsible department initiates corrective actions within 5 working days according to the "Nonconformity and Corrective Action Procedure"; for "Concern" conclusions, preventive improvement plans are developed.
• Management Review Input: The Quality Department includes the annual data analysis summary as one of the inputs for management reviews, submitting it to the General Manager for deliberation.
• Target Revision: Data analysis conclusions serve as the basis for setting and adjusting quality targets for the next year.
• Supplier Management: Supplier performance analysis results are used for supplier grading, elimination, and coaching according to the "Procurement and External Provider Control Procedure."
• Training Needs: Skill gaps identified through data analysis are fed back to the "Human Resources Management Procedure" as training needs.

4.5.2 Each department must clearly define the responsible person and completion timeline for improvement actions involving their department and provide timely feedback. The Quality Department tracks and verifies the effectiveness of these actions, recording the verification results in the "Corrective Action Handling Form."

4.6 Data Analysis Report

4.6.1 The Quality Department compiles the "Monthly Data Analysis Report" by the 15th of each month, covering the following content: completion status of various indicators, year-over-year and month-over-month comparisons, primary issue analysis using Pareto Charts, trend determination conclusions, and tracking of improvement actions. The annual data analysis report is compiled by the end of December and included in the management review.

4.6.2 The report is reviewed by the Management Representative and approved by the General Manager before being distributed to relevant departments. The monthly report should be issued to department heads within 2 working days, and significant abnormal situations should be reported immediately.

4.7 Flowchart (Text Version)

Departments collect raw data → Report data to the Quality Department according to the cycle → Quality Department verifies completeness
→ Select statistical methods for analysis → Compare and evaluate against target/historical/capability benchmarks
→ Determine trends (Good/Normal/Concern/Abnormal) → Formulate analysis report
→ Normal: Archive for reference; Concern: Develop preventive measures; Abnormal: Initiate corrective actions
→ Results input into management review, target revision, supplier management, and training needs
→ Quality Department tracks and verifies the effectiveness of improvement actions → Close the loop and archive

5. Related Records

Note: The record numbering rules follow the "Record Control Procedure," and each company can adjust the numbering system to fit its own needs.

6. Related Documents

• ISO 9001:2015 "Quality Management System Requirements" clause 9.1.3;
• "Record Control Procedure";
• "Quality Policy and Quality Target Management Procedure";
• "Customer Satisfaction Monitoring and Measurement Procedure";
• "Nonconforming Product Control Procedure";
• "Nonconformity and Corrective Action Procedure";
• "Procurement and External Provider Control Procedure";
• "Internal Audit Procedure";
• "Management Review Procedure."

Usage Instructions

1. How to Modify According to Actual Enterprise Conditions

1. Organizational Adaptation: Adjust the responsibilities table according to the departmental structure of the enterprise. In small enterprises, departments can be merged (e.g., the Quality Department can manage data collection), but the four key stages of "collection—analysis—application—tracking" must have clearly defined responsible positions to avoid situations where "data is not analyzed or conclusions are not implemented."
2. Product and Industry Adaptation: Manufacturing enterprises can focus on inspection pass rates, Cpk, and PPM as core indicators; service enterprises can adjust indicators to include customer complaint rates, service response times, first-time resolution rates, and satisfaction scores; construction projects can add indicators such as first-time pass rates for final inspections and accident rates. In section 4.3, retain 3-5 statistical tools based on actual usage capabilities, rather than introducing all of them.
3. Frequency and Scale Adaptation: For enterprises with small production scales and limited data volumes, monthly analysis can be changed to quarterly analysis, but the annual analysis report must be retained. When data volumes are insufficient to create control charts, focus on trend charts and descriptive statistics to avoid using tools for the sake of using them.
4. Information System Adaptation: Enterprises that have implemented ERP/MES/QMS systems can automatically aggregate data and generate reports through these systems. This procedure should supplement the rules for data extraction and the responsible persons for data, simplifying paper-based ledgers.

2. Audit Focus Points

1. Auditors will verify two lines: "whether there is data" and "whether the data is used." They will check both the original data from inspection records and complaint logs, as well as the data analysis reports and whether the conclusions have been converted into improvement actions. It is recommended to maintain a complete evidence chain of "data—analysis—actions—verification."
2. Common questions: Has the reason for the decrease in nonconforming rates been analyzed? How do the analysis results enter management reviews? Are supplier performance data used for supplier grading? Ensure that each conclusion can be traced to the corresponding action record.
3. Pay attention to the linkage evidence between 9.1.3 and 9.1.2, 9.2, 9.3, and 10.2: the distribution analysis of internal audit nonconformities, the data analysis section in management review inputs, and the correspondence between corrective actions and data analysis conclusions. These are frequent verification points in recent audits.

3. Common Errors

1. Only Statistics, No Analysis: Reports are limited to listing numbers like "98% pass rate" without trend determination, cause analysis, or improvement suggestions, leading to nonconformity in the "evaluation" requirement.
2. Inconsistent Data Definitions: Different departments have inconsistent definitions of "pass rate" and "complaints," leading to inaccurate aggregated data. The calculation formulas and statistical criteria for each indicator should be standardized in the procedure.
3. Disconnection Between Analysis and Management Review: Analysis reports are shelved after compilation and not included in management review inputs. It is recommended to list "data analysis conclusions and the implementation of actions" as a fixed input item in the management review procedure.
4. Misuse of Statistical Tools: Using control charts with insufficient sample sizes, or failing to distinguish between primary and secondary issues in Pareto Charts, can lead to misleading conclusions. Analysis personnel should be trained and retain training records.

Data speaks, analysis sets the direction

Knowledge code: 2.3.1

Version: v20260809

Author: QTank QTank is dedicated to providing systematic professional knowledge, methodologies, and practical tools for quality management practitioners, helping enterprises continuously improve their quality capabilities.