datanalysis-credit-risk

Credit risk data cleaning and variable screening pipeline for pre-loan modeling. The data cleaning pipeline consists of the following 11 steps, each executed independently without deleting the original data: The generated Excel report contains the following sheets: - Executes 11 independent steps covering data loading, abnormal period filtering, missing rate analysis, low-IV and high-PSI variable removal, null importance denoising, and correlation-based feature elimination - Supports organization-level analysis with separate modeling and out-of-sample (OOS) sample handling, plus multi-process acceleration for IV and PSI calculations - Generates comprehensive Excel report with 15 sheets detailing operation results, feature statistics, distributions, and removed variables across all pipeline stages - Configurable thresholds for missing rate, IV, PSI, correlation, and null importance parameters with sensible defaults - Get Data - Load and format raw data - Organization Sample Analysis - Statistics of sample count and bad sample rate for each organization - Separate OOS Data - Separate out-of-sample (OOS) samples from modeling samples - Filter Abnormal Months - Remove months with insufficient bad sample count or total sample count - Calculate Missing Rate - Calculate overall and organization-level missing rates for each feature - Drop High Missing Rate Features - Remove features...