wisski_database_checker/WISSKI-UPDATE-TESTING.md

WissKI Table Update Data Integrity Testing

This directory contains scripts to test data integrity when updating WissKI tables. The scripts ensure that no data is lost or truncated during schema updates.

Files

• test-wisski-data-integrity.php - PHP script to export and verify data
• test-wisski-update.sh - Bash wrapper script for the complete testing process

Usage

Option 1: Automated Testing (Recommended)

Run the complete test process (export → update → verify):

./test-wisski-update.sh

This script will:

1. Export all WissKI table data (row counts, checksums, sample rows)
2. Prompt you to run Drupal updates
3. Verify data integrity after updates
4. Generate a detailed report

Option 2: Manual Step-by-Step

Step 1: Export Data Before Update

php test-wisski-data-integrity.php export

This creates a directory wisski-data-integrity-test/ with:

• before-update-summary.json - Summary of all tables
• Individual table JSON files with row counts, checksums, and sample data

Step 2: Run Drupal Updates

drush updatedb -y

Or use the Drupal admin interface: /update.php

Step 3: Verify Data Integrity

php test-wisski-data-integrity.php verify

This compares the current state with the exported data and reports:

• Row count changes
• Checksum mismatches (data changes)
• Sample data differences

Output

All test results are saved in wisski-data-integrity-test/:

• before-update-summary.json - Complete export before update
• verification-report.json - Detailed verification results
• Individual table JSON files for detailed inspection

What Gets Tested

For each WissKI table (all tables with wisski prefix):

1. Row Count - Ensures no rows are lost or added unexpectedly
2. Data Checksum - MD5 hash of all table data to detect any changes
3. Sample Rows - First 10 rows compared to detect data corruption
4. Column Information - Schema changes are logged

Interpreting Results

✓ Success

• All row counts match
• All checksums match
• No data loss detected

⚠️ Issues Detected

• Row count mismatch: Rows were added or deleted
• Checksum mismatch: Data content changed (may be expected for schema updates)
• Sample data mismatch: First rows differ (investigate for corruption)

Important Notes

1. Backup First: Always backup your database before running updates
2. Schema Changes: Some checksum changes may be expected if column types change (e.g., VARCHAR to TEXT)
3. Large Tables: For very large tables, checksum calculation may fall back to partial checksums
4. Empty Tables: Empty tables are marked with checksum 'empty' and skipped in checksum comparison

Troubleshooting

If verification fails:

1. Check verification-report.json for detailed information
2. Compare individual table JSON files before/after
3. Review Drupal update logs for errors
4. Check if schema changes are expected (column type changes, etc.)

Example Output

Exporting data for 450 WissKI tables...
  Processing wisski_entity_map...
  Processing wisski_calling_bundles...
  ...

✓ Data export complete.

Verifying data integrity for 450 tables...
  Checking wisski_entity_map...
    ✓ Row count matches: 1234
    ✓ Checksum matches
    ✓ Sample data matches

==========================================
✓ SUCCESS: All data verified successfully!
==========================================