122 lines
3.4 KiB
Markdown
122 lines
3.4 KiB
Markdown
# WissKI Table Update Data Integrity Testing
|
|
|
|
This directory contains scripts to test data integrity when updating WissKI tables. The scripts ensure that no data is lost or truncated during schema updates.
|
|
|
|
## Files
|
|
|
|
- `test-wisski-data-integrity.php` - PHP script to export and verify data
|
|
- `test-wisski-update.sh` - Bash wrapper script for the complete testing process
|
|
|
|
## Usage
|
|
|
|
### Option 1: Automated Testing (Recommended)
|
|
|
|
Run the complete test process (export → update → verify):
|
|
|
|
```bash
|
|
./test-wisski-update.sh
|
|
```
|
|
|
|
This script will:
|
|
1. Export all WissKI table data (row counts, checksums, sample rows)
|
|
2. Prompt you to run Drupal updates
|
|
3. Verify data integrity after updates
|
|
4. Generate a detailed report
|
|
|
|
### Option 2: Manual Step-by-Step
|
|
|
|
#### Step 1: Export Data Before Update
|
|
|
|
```bash
|
|
php test-wisski-data-integrity.php export
|
|
```
|
|
|
|
This creates a directory `wisski-data-integrity-test/` with:
|
|
- `before-update-summary.json` - Summary of all tables
|
|
- Individual table JSON files with row counts, checksums, and sample data
|
|
|
|
#### Step 2: Run Drupal Updates
|
|
|
|
```bash
|
|
drush updatedb -y
|
|
```
|
|
|
|
Or use the Drupal admin interface: `/update.php`
|
|
|
|
#### Step 3: Verify Data Integrity
|
|
|
|
```bash
|
|
php test-wisski-data-integrity.php verify
|
|
```
|
|
|
|
This compares the current state with the exported data and reports:
|
|
- Row count changes
|
|
- Checksum mismatches (data changes)
|
|
- Sample data differences
|
|
|
|
## Output
|
|
|
|
All test results are saved in `wisski-data-integrity-test/`:
|
|
|
|
- `before-update-summary.json` - Complete export before update
|
|
- `verification-report.json` - Detailed verification results
|
|
- Individual table JSON files for detailed inspection
|
|
|
|
## What Gets Tested
|
|
|
|
For each WissKI table (all tables with `wisski` prefix):
|
|
|
|
1. **Row Count** - Ensures no rows are lost or added unexpectedly
|
|
2. **Data Checksum** - MD5 hash of all table data to detect any changes
|
|
3. **Sample Rows** - First 10 rows compared to detect data corruption
|
|
4. **Column Information** - Schema changes are logged
|
|
|
|
## Interpreting Results
|
|
|
|
### ✓ Success
|
|
- All row counts match
|
|
- All checksums match
|
|
- No data loss detected
|
|
|
|
### ⚠️ Issues Detected
|
|
- **Row count mismatch**: Rows were added or deleted
|
|
- **Checksum mismatch**: Data content changed (may be expected for schema updates)
|
|
- **Sample data mismatch**: First rows differ (investigate for corruption)
|
|
|
|
## Important Notes
|
|
|
|
1. **Backup First**: Always backup your database before running updates
|
|
2. **Schema Changes**: Some checksum changes may be expected if column types change (e.g., VARCHAR to TEXT)
|
|
3. **Large Tables**: For very large tables, checksum calculation may fall back to partial checksums
|
|
4. **Empty Tables**: Empty tables are marked with checksum 'empty' and skipped in checksum comparison
|
|
|
|
## Troubleshooting
|
|
|
|
If verification fails:
|
|
|
|
1. Check `verification-report.json` for detailed information
|
|
2. Compare individual table JSON files before/after
|
|
3. Review Drupal update logs for errors
|
|
4. Check if schema changes are expected (column type changes, etc.)
|
|
|
|
## Example Output
|
|
|
|
```
|
|
Exporting data for 450 WissKI tables...
|
|
Processing wisski_entity_map...
|
|
Processing wisski_calling_bundles...
|
|
...
|
|
|
|
✓ Data export complete.
|
|
|
|
Verifying data integrity for 450 tables...
|
|
Checking wisski_entity_map...
|
|
✓ Row count matches: 1234
|
|
✓ Checksum matches
|
|
✓ Sample data matches
|
|
|
|
==========================================
|
|
✓ SUCCESS: All data verified successfully!
|
|
==========================================
|
|
```
|
|
|