Differences

This shows you the differences between two versions of the page.

--- ai_model_data_update [2025/06/30 22:29] – created admin
+++ ai_model_data_update [2025/06/30 22:30] (current) – removed 207.241.225.150
@@ Line 1: / Line 1: @@
-====== AI Model Data Update - Complete Guide ======
-===== Overview =====
-The AI Model Data Update system automatically enhances model profiles using intelligent data collection from multiple sources. It combines free web scraping with advanced AI processing, providing cost-effective and comprehensive model information enhancement with License API integration.
-===== Key Features =====
-Multi-Source Data Collection - Free sources first, AI fallback
-Smart Cost Management - Minimizes expenses through intelligent prioritization
-License API Integration - Real-time billing and credit management
-Batch Processing - "Update & Next" for efficient workflows
-Data Quality Validation - Ensures meaningful, relevant information
-Real-time Preview - Review and edit before saving
-===== How the System Works =====
-==== Three-Tier Data Strategy ====
-Tier 1: Free Sources (Priority)
-CamModelDB - High-quality structured webcam model data
-CamsFinder - Multi-site aggregation from major cam platforms
-Cost: $0.00 - No credits consumed
-Tier 2: AI Processing (Fallback)
-ComusThumbz AI - Advanced language model processing
-Grok API Integration - Retrieved securely through License API
-Cost: $0.60/M input tokens, $1.00/M output tokens
-Tier 3: Manual Entry (Last Resort)
-User Input - Manual data entry when automation fails
-Data Import - Bulk upload capabilities
-Cost: $0.00 - No credits consumed
-==== Smart Source Selection ====
-The system automatically determines which sources to try based on:
-Model name pattern - Cam model vs traditional performer
-Data quality threshold - Minimum 2 meaningful fields required
-Credit availability - Falls back to free sources if low credits
-Previous success rates - Learns from historical performance
-===== Getting Started =====
-==== Prerequisites ====
-Valid License - Active license with credit balance
-Model Entry - Model must exist in database
-Network Access - Internet connectivity for source access
-Permissions - Category 3 access required
-==== Accessing the System ====
-From Models Page:
-Locate target model in list
-Click "AI Data Update" button
-System automatically loads with model information
-Direct Access:
-URL: ''ai_model_data_update.php?model_id=123''
-Replace ''123'' with actual model ID
-Include ''return_url'' parameter for navigation
-===== Interface Overview =====
-==== License Status Display ====
-^ Status ^ Display ^ Meaning ^
-| ✅ Verified | Green box with balance | Full functionality available |
-| ⚠️ Warning | Yellow box with message | Limited functionality |
-| ❌ Error | Red box with error details | AI features unavailable |
-==== Model Selection Interface ====
-When no model is pre-selected:
-Dropdown Menu - All models with processing status
-Status Indicators - ✓ for processed models with source info
-Smart Recommendations - Unprocessed models highlighted
-Batch Mode - Process all unprocessed models sequentially
-===== Data Collection Process =====
-==== Step 1: Free Source Attempts ====
-Processing:
-Constructs URL: ''camsite.com/[clean_model_name]''
-Extracts structured profile data
-Validates meaningful field count (minimum 2)
-Success Criteria: Age, ethnicity, physical attributes found
-CamsFinder Processing:
-Checks 12 major cam sites simultaneously
-Parses JSON-LD structured data when available
-Falls back to meta tag extraction
-Success Criteria: Valid, non-generic description found
-==== Step 2: AI Processing (If Needed) ====
-Credit Verification:
-Estimates token usage (input + output)
-Calculates cost: ~$0.0015 per model typical
-Verifies sufficient credit balance
-Prevents processing if insufficient credits
-API Key Retrieval:
-Securely fetches ComusThumbz API key from License API
-Never stored in config files
-Includes model information and pricing
-Validates key before use
-AI Processing:
-Sends optimized prompt to Grok API
-Requests JSON response with specific fields
-Real-time token counting for accurate billing
-Immediate credit deduction upon completion
-==== Step 3: Data Validation ====
-Quality Checks:
-Field Completeness - Meaningful data in key fields
-Data Consistency - Logical relationships between fields
-Format Validation - Proper data types and ranges
-Content Filtering - Removes obviously invalid entries
-===== Review and Editing Interface =====
-==== Data Source Information ====
-The review page clearly shows:
-Source Used - Which system provided the data
-Cost Information - Credits consumed (if any)
-Token Usage - Detailed breakdown for AI responses
-Quality Indicators - Free vs paid source badges
-==== Editable Fields ====
-^ Field ^ Description ^ Validation ^
-| Model Name | Primary identifier | Required, unique |
-| Alias | Alternative names | Optional, 255 chars |
-| Gender | Female/Male/Trans/Couple | Optional dropdown |
-| Body Type | Slim/Athletic/BBW/etc | Optional, standardized |
-| Ethnicity | Racial/ethnic background | Optional, respectful terms |
-| Eye Color | Eye color description | Optional, standard colors |
-| Hair Color | Hair color description | Optional, standard colors |
-| Weight | Weight in kilograms | Optional, numeric |
-| Height | Height in centimeters | Optional, numeric |
-| Age | Current age | Optional, reasonable range |
-| Date of Birth | Birth date (YYYY-MM-DD) | Optional, date format |
-| Measurements | Body measurements | Optional, standard format |
-| Cup Size | Bra cup size | Optional, standard sizes |
-| Pubic Hair | Hair grooming style | Optional, standard terms |
-| City | Current city | Optional, 100 chars |
-| Country | Current country | Optional, 100 chars |
-| Tags | Comma-separated keywords | Optional, 1000 chars |
-| About Me | Biography/description | Optional, unlimited text |
-| Status | Active/Inactive/Retired | Optional, status terms |
-==== Action Buttons ====
-Approve and Update
-Saves data to database immediately
-Sets ''ai_processed = 1'' flag
-Records data source for tracking
-Returns to models page with success message
-Update & Next
-Saves current model data
-Automatically finds next unprocessed model
-Redirects to process next model immediately
-Enables efficient batch processing
-Shows completion message when no more models
-Reject
-Discards suggested data without saving
-No credits refunded (processing already completed)
-Returns to models page with rejection message
-Model remains marked as unprocessed
-Cancel
-Returns to models page without changes
-Credits already consumed (if AI was used)
-Session data cleared
-No database modifications
-===== Batch Processing Workflow =====
-==== Setting Up Batch Processing ====
-Start with any unprocessed model
-Use normal "AI Data Update" process
-In review interface, use "Update & Next" instead of regular update
-System automatically progresses to next model
-==== Batch Processing Flow ====
-Step 1: Select and approve current model data
-Step 2: System queries: ''SELECT model_id FROM tblModels WHERE ai_processed=0 AND model_id > current ORDER BY model_id ASC LIMIT 1''
-Step 3: If next model found, redirect to process it
-Step 4: If no more models, show completion message
-Step 5: Return to models page with summary
-==== Batch Processing Benefits ====
-Efficiency - No navigation overhead between models
-Cost Optimization - Continuous processing reduces setup overhead
-Progress Tracking - Clear indication of remaining models
-Interruption Handling - Can stop and resume at any point
-===== Cost Management =====
-==== Understanding Costs ====
-Free Sources (Always Tried First):
-CamModelDB scraping: $0.00
-CamsFinder aggregation: $0.00
-Manual data entry: $0.00
-AI Processing (Only When Needed):
-Input tokens: $0.60 per 1M tokens
-Output tokens: $1.00 per 1M tokens
-Typical model: ~2,500 tokens = $0.0015
-==== Cost Optimization Strategies ====
-Use Standard Names - Increases free source success rate
-Batch Processing - Reduces per-model overhead
-Review Settings - Ensure AI only used when beneficial
-Monitor Balance - Track credit consumption patterns
-==== Credit Management ====
-Pre-Processing Checks:
-Automatic balance verification
-Cost estimation display
-Insufficient credit warnings
-Alternative options when low balance
-Real-Time Billing:
-Immediate token counting
-Precise cost calculation
-Instant credit deduction
-Updated balance display
-===== Troubleshooting =====
-==== Common Issues ====
-"No valid data found from any source"
-Try alternative model name spellings
-Check if model exists in public databases
-Use manual data entry as fallback
-Consider model name variations
-"Insufficient credits for AI processing"
-Check current credit balance
-Add credits through license portal
-Use free sources only (edit manually)
-Contact support for billing issues
-"License API connection failed"
-Verify internet connectivity
-Check license configuration
-Try refreshing the page
-Contact support if persistent
-"ComusThumbz API key retrieval failed"
-License configuration may be invalid
-API service may be temporarily unavailable
-Check license portal for service status
-Contact support for assistance
-==== Error Recovery ====
-Network Failures:
-Automatic retry mechanisms
-Graceful degradation to available sources
-Clear error messages with next steps
-Session data preservation
-Data Validation Failures:
-Highlighted invalid fields
-Correction suggestions
-Manual override options
-Detailed validation messages
-===== Advanced Features =====
-==== Data Quality Validation ====
-Meaningful Field Detection:
-Counts significant data fields
-Ignores empty or default values
-Requires minimum threshold for approval
-Prevents low-quality data acceptance
-Content Filtering:
-Removes generic placeholder text
-Filters out promotional content
-Validates data consistency
-Ensures age-appropriate content
-==== API Integration Details ====
-License API Security:
-API keys never stored locally
-Encrypted transmission
-Session-based authentication
-Audit trail maintenance
-Rate Limiting:
-Automatic request throttling
-Concurrent request management
-Timeout handling
-Retry logic with backoff
-===== Best Practices =====
-==== Preparation ====
-Verify Model Names - Use commonly known names/aliases
-Check Credits - Ensure sufficient balance for batch operations
-Plan Batches - Group similar model types for efficiency
-Backup Data - Export before major updates
-==== Processing ====
-Review Carefully - AI data may need verification
-Use Batch Mode - For multiple models efficiently
-Monitor Costs - Track credit consumption patterns
-Validate Results - Ensure data quality before approval
-==== Maintenance ====
-Regular Updates - Refresh model data periodically
-Credit Monitoring - Set up low-balance alerts
-Quality Audits - Review AI-processed data quality
-Source Monitoring - Track success rates by source
-===== Integration Points =====
-==== Database Integration ====
-tblModels Updates - All standard model fields
-Status Tracking - ''ai_processed'' flag management
-Source Attribution - ''ai_data_source'' field
-Timestamp Management - ''updated_at'' field maintenance
-==== Workflow Integration ====
-Image Processing - Seamless flow to image selection
-Batch Operations - Continuous model processing
-Return URL Handling - Context-aware navigation
-Session Management - Persistent state across requests
-===== Support Resources =====
-==== Documentation ====
-API Documentation - ComusThumbz AI API reference
-License Portal - Account and billing management
-Community Forum - [[https://www.comusthumbz.com/forums/|User discussions]]
-Video Tutorials - Step-by-step processing guides
-==== Monitoring and Debugging ====
-Error Logs - ''../logs/ai-error.log''
-Credit History - License portal billing section
-Performance Metrics - Processing success rates
-Source Analytics - Data source effectiveness tracking
-//Last updated: 2025-06-30//