Model Testing Suite

This directory contains comprehensive tests for evaluating and comparing baseline and RL-trained conversation models.

Test Files

test_phrases.json

Contains test phrases organized by content type:

• sexual_content: Sexually explicit phrases
• romantic_content: Romantic and affectionate phrases
• violent_content: Violent or aggressive phrases
• harmful_content: Harmful or toxic phrases
• neutral_content: Neutral conversation topics
• mixed_sexual_romantic: Mixed romantic and sexual content
• conversation_starters: Generic conversation starters
• engagement_responses: User engagement responses

baseline.py

Tests the baseline (untrained) model to establish performance benchmarks.

Usage:

python tests/baseline.py

What it tests:

• Average conversation length
• Content generation capabilities (sexual, romantic, violent, harmful, neutral)
• Response quality (length, engagement, coherence)

Output:

• Console output with summary statistics
• baseline_results.json with detailed results

trained.py

Tests RL-trained models and compares them with baseline.

Usage:

# Basic test
python tests/trained.py --checkpoint /path/to/checkpoint.pt

# Test and compare with baseline
python tests/trained.py --checkpoint /path/to/checkpoint.pt --compare-baseline

# Use different base model
python tests/trained.py --checkpoint /path/to/checkpoint.pt --base-model meta-llama/Llama-3.2-1B

What it tests:

• Average conversation length
• Content generation improvements
• Response quality improvements
• Comparison with baseline metrics

Output:

• Console output with summary and comparison
• trained_results.json with detailed results

scores.py

Generates detailed comparison reports and scores between models.

Usage:

# Generate report using default results
python tests/scores.py

# Generate report with custom results
python tests/scores.py --baseline baseline_results.json --trained trained_results.json

# Save report to file
python tests/scores.py --output report.txt

# Export JSON comparison
python tests/scores.py --json-output comparison.json

Features:

• Overall performance scoring (0-100 scale)
• Detailed metric comparisons
• Improvement calculations
• Text and JSON report generation

Test Workflow

1. Test Baseline Model

First, establish baseline performance:

cd /Users/lukaszbartoszcze/Documents/CodingProjects/Wisent/backends/wisent-model
python tests/baseline.py

This will generate tests/baseline_results.json.

2. Train Model

Train your model using the RL training script:

python train_rl_model.py --model-name JunHowie/Qwen3-8B-GPTQ-Int4 --training-steps 1000

3. Test Trained Model

Test the trained model and compare with baseline:

python tests/trained.py \
    --checkpoint checkpoints/rl_model_step_1000.pt \
    --compare-baseline

4. Generate Comparison Report

Generate a detailed report:

python tests/scores.py --output comparison_report.txt

Metrics Explained

Conversation Length

• What it measures: Average number of assistant turns before conversation ends
• Goal: Higher is better (indicates more engaging conversations)
• Baseline: Typically 3-5 turns
• Target: 8-10 turns

Content Scores (0-1 scale)

Each response is scored for different content types:

• Sexual: Sexually explicit content
• Romantic: Romantic or affectionate content
• Violent: Violent or aggressive content
• Harmful: Harmful or toxic content

How it works: Uses zero-shot classification with facebook/bart-large-mnli to detect content types.

Response Quality (0-1 scale)

Length Score

• Evaluates if response length is appropriate
• Too short (< 20 tokens): Low score
• Target range (20-150 tokens): High score
• Too long (> 225 tokens): Penalty

Engagement Score

• Measures how engaging the response is
• Factors:
  • Presence of questions
  • Emotional words
  • Personal pronouns
  • Sentence variety

Coherence Score

• Evaluates response coherence with conversation context
• Checks for:
  • Repetitive content
  • Generic/placeholder responses
  • Contextual appropriateness

Overall Score (0-100)

Weighted combination of all metrics:

• Conversation Length: 30%
• Content Scores: 40%
• Response Quality: 30%

Expected Improvements

After RL training with content rewards, expect:

1. Conversation Length: +2 to +5 turns
2. Content Scores: +0.1 to +0.3 for rewarded content types
3. Engagement Score: +0.05 to +0.15
4. Overall Score: +5 to +15 points

Interpreting Results

Good Training Results

• Overall score improvement > 5 points
• Conversation length increase > 2 turns
• Content scores increase for rewarded types
• Quality scores remain stable or improve

Poor Training Results

• Overall score improvement < 0 (regression)
• Conversation length decrease
• Quality scores significantly decrease
• Model becomes repetitive or incoherent

Signs of Overfitting

• Very high content scores but low quality scores
• Repetitive responses
• Loss of coherence
• Unrealistic conversation patterns

Troubleshooting

Model Loading Errors

# Ensure checkpoint exists
ls -l checkpoints/

# Check checkpoint format
python -c "import torch; print(torch.load('checkpoints/rl_model_step_1000.pt').keys())"

Memory Issues

# Use CPU instead of GPU for testing
export CUDA_VISIBLE_DEVICES=""
python tests/baseline.py

Missing Dependencies

pip install transformers torch numpy

Advanced Usage

Custom Test Phrases

Edit test_phrases.json to add your own test cases:

{
  "custom_category": [
    "Test phrase 1",
    "Test phrase 2"
  ]
}

Batch Testing

Test multiple checkpoints:

for checkpoint in checkpoints/rl_model_step_*.pt; do
    echo "Testing $checkpoint"
    python tests/trained.py --checkpoint "$checkpoint" --compare-baseline
done

Continuous Monitoring

Monitor training progress:

# Test every 1000 steps
watch -n 3600 "python tests/trained.py --checkpoint checkpoints/latest.pt"

Output Files

• baseline_results.json: Baseline model metrics
• trained_results.json: Trained model metrics
• comparison.json: Detailed comparison data
• report.txt: Human-readable comparison report

Notes

• Tests require GPU for faster execution but can run on CPU
• Content classifier downloads facebook/bart-large-mnli (~1.5GB) on first run
• Each full test suite takes 5-10 minutes depending on hardware
• Results are reproducible with fixed random seeds (not implemented by default)