first commit

WISSKI-UPDATE-TESTING.md

@@ -0,0 +1,122 @@
+# WissKI Table Update Data Integrity Testing
+
+This directory contains scripts to test data integrity when updating WissKI tables. The scripts ensure that no data is lost or truncated during schema updates.
+
+## Files
+
+- `test-wisski-data-integrity.php` - PHP script to export and verify data
+- `test-wisski-update.sh` - Bash wrapper script for the complete testing process
+
+## Usage
+
+### Option 1: Automated Testing (Recommended)
+
+Run the complete test process (export → update → verify):
+
+```bash
+./test-wisski-update.sh
+```
+
+This script will:
+1. Export all WissKI table data (row counts, checksums, sample rows)
+2. Prompt you to run Drupal updates
+3. Verify data integrity after updates
+4. Generate a detailed report
+
+### Option 2: Manual Step-by-Step
+
+#### Step 1: Export Data Before Update
+
+```bash
+php test-wisski-data-integrity.php export
+```
+
+This creates a directory `wisski-data-integrity-test/` with:
+- `before-update-summary.json` - Summary of all tables
+- Individual table JSON files with row counts, checksums, and sample data
+
+#### Step 2: Run Drupal Updates
+
+```bash
+drush updatedb -y
+```
+
+Or use the Drupal admin interface: `/update.php`
+
+#### Step 3: Verify Data Integrity
+
+```bash
+php test-wisski-data-integrity.php verify
+```
+
+This compares the current state with the exported data and reports:
+- Row count changes
+- Checksum mismatches (data changes)
+- Sample data differences
+
+## Output
+
+All test results are saved in `wisski-data-integrity-test/`:
+
+- `before-update-summary.json` - Complete export before update
+- `verification-report.json` - Detailed verification results
+- Individual table JSON files for detailed inspection
+
+## What Gets Tested
+
+For each WissKI table (all tables with `wisski` prefix):
+
+1. **Row Count** - Ensures no rows are lost or added unexpectedly
+2. **Data Checksum** - MD5 hash of all table data to detect any changes
+3. **Sample Rows** - First 10 rows compared to detect data corruption
+4. **Column Information** - Schema changes are logged
+
+## Interpreting Results
+
+### ✓ Success
+- All row counts match
+- All checksums match
+- No data loss detected
+
+### ⚠️ Issues Detected
+- **Row count mismatch**: Rows were added or deleted
+- **Checksum mismatch**: Data content changed (may be expected for schema updates)
+- **Sample data mismatch**: First rows differ (investigate for corruption)
+
+## Important Notes
+
+1. **Backup First**: Always backup your database before running updates
+2. **Schema Changes**: Some checksum changes may be expected if column types change (e.g., VARCHAR to TEXT)
+3. **Large Tables**: For very large tables, checksum calculation may fall back to partial checksums
+4. **Empty Tables**: Empty tables are marked with checksum 'empty' and skipped in checksum comparison
+
+## Troubleshooting
+
+If verification fails:
+
+1. Check `verification-report.json` for detailed information
+2. Compare individual table JSON files before/after
+3. Review Drupal update logs for errors
+4. Check if schema changes are expected (column type changes, etc.)
+
+## Example Output
+
+```
+Exporting data for 450 WissKI tables...
+  Processing wisski_entity_map...
+  Processing wisski_calling_bundles...
+  ...
+
+✓ Data export complete.
+
+Verifying data integrity for 450 tables...
+  Checking wisski_entity_map...
+    ✓ Row count matches: 1234
+    ✓ Checksum matches
+    ✓ Sample data matches
+
+==========================================
+✓ SUCCESS: All data verified successfully!
+==========================================
+```
+