Overview

The Conversion Servers page provides a centralized interface for managing distributed video processing infrastructure. This system allows you to offload CPU-intensive video conversion tasks (transcoding, HLS generation, thumbnail creation) to remote servers, dramatically improving processing throughput and reducing load on your main web server.

Key Features

• Multi-Server Support - Connect unlimited conversion servers via SSH/SFTP, FTP, Local, or S3
• Real-Time Monitoring - Dashboard with live stats, health indicators, and activity logs
• Automatic Load Balancing - Distribute tasks across servers using configurable algorithms
• Worker Deployment - One-click deployment of worker scripts to remote servers
• Health Monitoring - Automatic heartbeat detection with configurable thresholds
• Task Management - View, retry, and manage conversion tasks across all servers
• Diagnostics - Built-in tools for troubleshooting connection and processing issues

Architecture Overview

┌─────────────────┐     ┌──────────────────────┐     ┌─────────────────┐
│   Main Server   │────▶│  ConversionServer    │────▶│ Remote Server 1 │
│  (Web Server)   │     │     Manager          │     │  (FFmpeg Node)  │
│                 │     │                      │     └─────────────────┘
│ conversion     │     │ - Load Balancing     │     ┌─────────────────┐
│ servers.php     │     │ - Task Assignment    │────▶│ Remote Server 2 │
│                 │     │ - Health Monitoring  │     │  (FFmpeg Node)  │
│ RemoteJob       │     │ - Result Collection  │     └─────────────────┘
│ Dispatcher      │     └──────────────────────┘     ┌─────────────────┐
└─────────────────┘                              ────▶│ Remote Server N │
                                                      │  (FFmpeg Node)  │
                                                      └─────────────────┘

Communication Flow:

Main server creates task and uploads source file to remote server

1. Remote worker (remoteworker.php) processes the file using FFmpeg
2. Worker writes progress updates to progress.dat file
3. Main server polls for completion, downloads results
4. Results stored on CDN/storage servers

System Requirements

PHP Extensions

At least one connection extension required based on your server types.

PHP Functions

External Tools (Remote Servers)

php.ini Settings

; Main server settings
maxexecutiontime = 300      ; Allow time for large file uploads
memorylimit = 512M           ; Handle large task queues
uploadmaxfilesize = 10G     ; Support large video uploads
postmaxsize = 10G           ; Match upload size

; SSH2 extension (if using SSH connections)
; Ensure ssh2.so is loaded in php.ini

; For long-running operations
settimelimit = 0 ; May be needed for worker deployment

Installation Requirements

Config Variables

Add these to ct/dat/config.inc.php:

// Conversion Server Settings
$conversionencryptionkey = 'your-32-character-encryption-key'; // CHANGE THIS!
$conversionheartbeatthreshold = 300; // Seconds before server marked unhealthy
$conversiontasktimeout = 3600; // Max task processing time (1 hour)
$conversionmaxretries = 3; // Maximum retry attempts for failed tasks
$conversionchunksize = 10485760; // 10MB chunks for file uploads
$conversionloadbalancemethod = 'leastloaded'; // roundrobin, leastloaded, weighted, priority

// Remote Worker Settings
$remoteworkercheckinterval = 5; // Seconds between progress checks
$remoteworkercleanupdays = 7; // Days to keep completed task files

Cron Jobs

# Process pending conversion tasks (every minute) - RECOMMENDED - ONLY ONE NEEDED, OTHERS ADDED FOR USERS THAT MAY ONLY USE ONE PART OF THE SCRIPT.

• php /path/to/ct/admin/cron/sitecron.php >> /path/to/ct/logs/conversion/cron.log 2>&1

Process pending conversion tasks (every minute)

• php /path/to/ct/admin/cron/conversiontaskprocessor.php >> /path/to/ct/logs/conversion/cron.log 2>&1

Health check all servers (every 5 minutes)

/5     php /path/to/ct/admin/cron/conversionhealthmonitor.php >> /path/to/ct/logs/conversion/health.log 2>&1

Cleanup old task files (daily at 3 AM)

0 3    php /path/to/ct/admin/cron/conversioncleanup.php >> /path/to/ct/logs/conversion/cleanup.log 2>&1

Retry failed tasks (every 15 minutes)

/15     php /path/to/ct/admin/cron/conversionretryhandler.php >> /path/to/ct/logs/conversion/retry.log 2>&1

Folder Structure & Permissions

ct/
├── admin/
│   ├── conversionservers.php          # Main admin page
│   ├── ConversionServerManager.php     # Server management class
│   ├── RemoteJobDispatcher.php         # Task dispatching class
│   ├── remoteworker.php               # Worker script (deployed to remote)
│   └── cron/
│       ├── conversiontaskprocessor.php
│       ├── conversionhealthmonitor.php
│       ├── conversioncleanup.php
│       └── conversionretryhandler.php
├── logs/
│   └── conversion/                     # chmod 755
│       ├── cron.log
│       ├── health.log
│       ├── tasks/                      # Per-task logs
│       └── servers/                    # Per-server logs
└── uploads/
    └── conversiontemp/                # chmod 755 - Temporary files

Remote Server Structure:

/path/to/remotepath/                   # Configured per server

├── remoteworker.php                   # Deployed worker script

├── config.php                          # Deployed configuration

├── tasks/

│   └── {taskid}/

│       ├── task.dat                    # Task definition

│       ├── progress.dat                # Progress updates

│       ├── result.dat                  # Completion results

│       ├── error.dat                   # Error information

│       ├── source/                     # Source files

│       └── output/                     # Processed files

└── logs/

    └── worker.log

User Interface

Dashboard Stats Grid

The page opens with a statistics dashboard showing real-time metrics:

Tab Navigation

The interface uses two main tabs:

Servers Tab

Table Columns:

Action Buttons:

Activity Logs Tab

Log Table Columns:

Log Filters:

• Level Filter: All, Debug, Info, Warning, Error, Critical
• Type Filter: All, Connection, Task, Health, Deployment, System
• Server Filter: All Servers, or specific server
• Date Range: From/To date pickers

Add/Edit Server Modal

The modal is divided into collapsible sections:

Connection Settings Section

Required for SSH/FTP types. S3 uses different fields.

S3-Specific Fields (shown when S3 selected):

Paths Configuration Section

Capacity Settings Section

Hardware Info Section (Optional)

Deploy Worker Modal

Deploys the worker script and configuration to a remote server.

Files Deployed:

1. remoteworker.php - The main worker script
2. config.php - Server-specific configuration

Diagnostics Modal

Displays detailed health information from the remote server's heartbeat.

Step-by-Step Usage

Adding a New SSH Server

1. Click "Add Server" button in the top-right corner

1. Fill Connection Settings:

• Server Name: Production-Server-1
• Server Type: SSH/SFTP
• Host: 192.168.1.100 or conversion1.yourdomain.com
• Port: 22 (default for SSH)
• Username: conversionuser
• Password: Your secure password

1. Configure Paths:

• Remote Path: /home/conversionuser/videotasks
• FFmpeg Path: /usr/bin/ffmpeg
• FFprobe Path: /usr/bin/ffprobe

1. Set Capacity:

• Max Concurrent Jobs: 4 (based on CPU cores)
• Priority: 50 (higher priority servers get tasks first)
• Enabled: Checked

1. Add Hardware Info (Optional):

• CPU Cores: 8
• RAM: 32
• Disk Space: 500

1. Click "Save Server"

1. Test Connection: Click the plug icon to verify connectivity

1. Deploy Worker: Click the upload icon to deploy worker files

Adding an FTP Server

1. Click "Add Server"

1. Fill Connection Settings:

Server Name: FTP-Converter-1Server Type: FTPHost: ftp.yourdomain.comPort: 21 (or 990 for FTPS)

• Username: ftpconversion
• Password: Your FTP password

1. Configure Paths:

• Remote Path: /publichtml/conversion
• FFmpeg Path: /usr/local/bin/ffmpeg
• FFprobe Path: /usr/local/bin/ffprobe

1. Save and Test

Adding an S3-Compatible Server

1. Click "Add Server"

1. Fill Connection Settings:

• Server Name: S3-Processing-Node
• Server Type: S3 Compatible

1. Configure S3 Settings:

• S3 Bucket: video-conversion-tasks
• S3 Region: us-east-1
• S3 Access Key: Your access key
• S3 Secret Key: Your secret key

1. Configure Paths:

• Remote Path: /tasks (path within bucket)

Deploying Worker Files

1. Select the server from the servers list

1. Click "Deploy Worker" (upload icon)

1. In the modal:

• Verify the correct server is selected
• Review files to be deployed
• Click "Deploy"

1. Monitor Progress:

• Progress bar shows upload status
• Status messages indicate each step
• Green checkmarks show successful uploads

1. Verification:

• System automatically tests worker after deployment
• Health status updates to "Healthy" if successful

Worker Files Structure on Remote:

/home/conversionuser/videotasks/

├── remoteworker.php      # Main processing script

├── config.php             # Server configuration

├── tasks/                 # Created automatically

└── logs/                  # Worker logs

Running Diagnostics

1. Click "Diagnostics" (stethoscope icon) on any server

1. Review System Information:

• PHP version compatibility
• FFmpeg/FFprobe availability
• Available disk space

1. Check Resource Usage:

• Current CPU load
• Memory utilization
• Active processes

1. Review Active Tasks:

• Tasks currently processing
• Progress percentages
• Estimated completion times

1. Examine Recent Errors:

• Connection failures
• Processing errors
• Timeout issues

Understanding Task Flow

┌─────────────┐
│   pending   │ ← Task created, waiting for server
└──────┬──────┘
       │
       ▼
┌─────────────┐
│  assigned   │ ← Server selected via load balancing
└──────┬──────┘
       │
       ▼
┌─────────────┐
│  uploading  │ ← Source file uploading to remote
└──────┬──────┘
       │
       ▼
┌─────────────┐
│   queued    │ ← File uploaded, waiting for worker
└──────┬──────┘
       │
       ▼
┌─────────────┐
│ processing  │ ← FFmpeg actively converting
└──────┬──────┘
       │
       ▼
┌─────────────┐
│ downloading │ ← Results being transferred back
└──────┬──────┘
       │
       ▼
┌─────────────┐
│  completed  │ ← Task finished successfully
└─────────────┘
       │
       ▼ (on error)
┌─────────────┐
│   failed    │ ← Error occurred, may retry
└─────────────┘

Monitoring Task Progress

1. Dashboard Stats show overall queue status

1. Activity Logs Tab displays:

• Task assignments
• Processing updates
• Completion/failure notifications

1. Diagnostics Modal shows:

• Active tasks on specific server
• Real-time progress percentages

1. Database Query for detailed status:

SELECT
    t.taskid,
    t.videoid,
    s.servername,
    t.status,
    t.progresspercent,
    t.progressmessage,
    t.createdat,
    t.processingstartedat
FROM tblConversionTasks t
LEFT JOIN tblConversionServers s ON t.serverid = s.serverid
WHERE t.status NOT IN ('completed', 'cancelled')
ORDER BY t.createdat DESC;

Load Balancing Methods

The system supports four load balancing strategies configured via $conversionloadbalancemethod:

Round Robin (roundrobin)

Distributes tasks evenly across all healthy servers in rotation.

Best For: Servers with identical hardware specs

Task 1 → Server A
Task 2 → Server B
Task 3 → Server C
Task 4 → Server A
...

Least Loaded (leastloaded)

Assigns tasks to the server with the lowest current load percentage.

Best For: Mixed workloads, general use (default)

// Load calculation
$loadpercent = ($currentjobs / $maxconcurrentjobs)  100;
// Server with lowest loadpercent gets the task

Weighted (weighted)

Uses server priority values as weights for random selection.

Best For: Servers with different capabilities

Server A: Priority 30 → 30% chance
Server B: Priority 50 → 50% chance
Server C: Priority 20 → 20% chance

Priority (priority)

Always selects the available server with highest priority.

Best For: Primary/backup server configurations

Server A: Priority 100 → Gets all tasks (if available)
Server B: Priority 50  → Only used if A is at capacity
Server C: Priority 10  → Emergency backup only

Troubleshooting

Connection Issues

SSH Connection Failed

Symptoms:

• "Connection refused" error
• "Authentication failed" error
• Timeout during connection test

Solutions:

# On remote server

sudo systemctl status sshd

# On remote server
sudo ufw status
sudo ufw allow 22/tcp

# Test from main server
ssh username@host -p port

# /etc/ssh/sshdconfig
PasswordAuthentication yes
PubkeyAuthentication yes

FTP Connection Failed

Symptoms:

• "Unable to connect to FTP server"
• "Login authentication failed"
• Passive mode errors

Solutions:

sudo systemctl status vsftpd  # or proftpd

// In your FTP client code
ftppasv($conn, true);

telnet host 21

# Ensure user can access and write to remotepath
ls -la /path/to/remotepath

S3 Connection Failed

Symptoms:

• "Access Denied" error
• "Bucket not found"
• Signature mismatch

Solutions:

{
    "Version": "2012-10-17",
    "Statement": [{
        "Effect": "Allow",
        "Action": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject"],
        "Resource": "arn:aws:s3:::your-bucket/"
    }]
}

Worker Deployment Issues

Worker Not Executing

Symptoms:

• Tasks stuck in "queued" status
• No progress updates
• Heartbeat missing

Solutions:

# On remote server

which php

php -v

chmod 755 /path/to/remoteworker.php
chmod 755 /path/to/tasks/

# Add to remote server's crontab

php /path/to/remoteworker.php

FFmpeg Not Found

Symptoms:

• "FFmpeg binary not found"
• "Command not found: ffmpeg"
• Processing immediately fails

Solutions:

which ffmpeg

whereis ffmpeg

# Ubuntu/Debian
sudo apt update && sudo apt install ffmpeg

sudo yum install epel-release
sudo yum install ffmpeg

Task Processing Issues

Tasks Stuck in Processing

Symptoms:

• Tasks show "processing" for extended time
• Progress percentage not updating
• No completion or error

Solutions:

tail -f /path/to/logs/worker.log

cat /path/to/tasks/{taskid}/progress.dat

ps aux | grep ffmpeg

pkill -f "ffmpeg.taskid"

$conversiontasktimeout = 7200; // 2 hours

Tasks Failing Immediately

Symptoms:

• Tasks go from "pending" to "failed"
• Error message: "Source file not found"
• Error message: "Insufficient disk space"

Solutions:

ls -la /path/to/source/video.mp4

df -h /path/to/tasks

# Worker user must read source
chmod 644 /path/to/source/video.mp4

chmod 755 /path/to/tasks/

SELECT errormessage, context
FROM tblConversionServerLogs
WHERE taskid = X
ORDER BY createdat DESC;

Health Monitoring Issues

All Servers Showing "Offline"

Symptoms:

• Dashboard shows all servers offline
• "Offline" count equals total servers
• Last heartbeat never updates

Solutions:

crontab -l | grep healthmonitor

php /path/to/ct/admin/cron/conversionhealthmonitor.php

cat /path/to/tasks/heartbeat.dat

# In remoteworker.php, heartbeat is written every cycle

$conversionheartbeatthreshold = 600; // 10 minutes

Intermittent "Degraded" Status

Symptoms:

• Server alternates between "healthy" and "degraded"
• Heartbeat times vary significantly
• High server load during transitions

Solutions:

top -bn1 | head -20

$conversionheartbeatthreshold = 600; // Allow 10 minute gaps

ping remoteserverip
traceroute remoteserverip

Security Considerations

Password Encryption

All passwords are encrypted using AES-256-CBC before storage:

// How passwords are encrypted
$encrypted = opensslencrypt(
    $password,
    'aes-256-cbc',
    $conversionencryptionkey,
    0,
    $iv
);

SSH Key Authentication

For enhanced security, use SSH key authentication instead of passwords:

1. Generate key pair on main server:

ssh-keygen -t ed25519 -f /path/to/conversionkey

1. Copy public key to remote server:

ssh-copy-id -i /path/to/conversionkey.pub user@remote

1. Configure server in admin:

• Leave password empty
• Set SSH Key Path: /path/to/conversionkey

Network Security

File Permission Best Practices

# Remote server permissions
chmod 700 /path/to/videotasks           # Only owner access
chmod 600 /path/to/videotasks/config.php # Protect config
chmod 755 /path/to/videotasks/tasks      # Allow task processing
chown -R conversionuser:conversionuser /path/to/videotasks

Performance Optimization

Optimal Server Configuration

Per Server Settings:

FFmpeg Optimization

For optimal transcoding speed, configure FFmpeg with hardware acceleration:

# Check for hardware acceleration support
ffmpeg -hwaccels

NVIDIA GPU acceleration

ffmpeg -hwaccel cuda -i input.mp4 ...

Intel Quick Sync

ffmpeg -hwaccel qsv -i input.mp4 ...

AMD VCE

ffmpeg -hwaccel vaapi -i input.mp4 ...

Network Optimization

For large file transfers:

1. Use compression for text-based protocols
2. Increase chunk size for stable connections:

$conversionchunksize = 52428800; // 50MB chunks

1. Enable jumbo frames if on private network
2. Use local storage servers for temporary files

API Reference

ConversionServerManager Methods

RemoteJobDispatcher Methods

Translatable Strings

Add these keys to lang/en.json:

{
  "conversionserverstitle": "Conversion Servers",
  "conversionserverssubtitle": "Manage distributed video processing nodes",
  "conversionaddserver": "Add Server",
  "conversioneditserver": "Edit Server",
  "conversiondeleteserver": "Delete Server",
  "conversiondeleteconfirm": "Are you sure you want to delete this server? This action cannot be undone.",
  "conversionservername": "Server Name",
  "conversionservertype": "Server Type",
  "conversionserverhost": "Host",
  "conversionserverport": "Port",
  "conversionserverusername": "Username",
  "conversionserverpassword": "Password",
  "conversionsshkeypath": "SSH Key Path",
  "conversionremotepath": "Remote Path",
  "conversionffmpegpath": "FFmpeg Path",
  "conversionffprobepath": "FFprobe Path",
  "conversionmaxjobs": "Max Concurrent Jobs",
  "conversionpriority": "Priority",
  "conversionenabled": "Enabled",
  "conversioncpucores": "CPU Cores",
  "conversionramgb": "RAM (GB)",
  "conversiondiskspace": "Disk Space (GB)",
  "conversions3bucket": "S3 Bucket",
  "conversions3region": "S3 Region",
  "conversions3accesskey": "S3 Access Key",
  "conversions3secretkey": "S3 Secret Key",
  "conversiontestconnection": "Test Connection",
  "conversiondeployworker": "Deploy Worker",
  "conversiondiagnostics": "Diagnostics",
  "conversionconnectionsettings": "Connection Settings",
  "conversionpathsettings": "Path Configuration",
  "conversioncapacitysettings": "Capacity Settings",
  "conversionhardwareinfo": "Hardware Info",
  "conversiontabservers": "Servers",
  "conversiontablogs": "Activity Logs",
  "conversionstattotal": "Total Servers",
  "conversionstatenabled": "Enabled",
  "conversionstathealthy": "Healthy",
  "conversionstatoffline": "Offline",
  "conversionstatload": "Current Load",
  "conversionstatpending": "Pending Tasks",
  "conversionstatcompleted": "Completed Today",
  "conversionstatfailed": "Failed Today",
  "conversionhealthhealthy": "Healthy",
  "conversionhealthdegraded": "Degraded",
  "conversionhealthoffline": "Offline",
  "conversionhealthunknown": "Unknown",
  "conversiontypessh": "SSH/SFTP",
  "conversiontypeftp": "FTP",
  "conversiontypelocal": "Local",
  "conversiontypes3": "S3 Compatible",
  "conversionconnectionsuccess": "Connection successful!",
  "conversionconnectionfailed": "Connection failed",
  "conversiondeploysuccess": "Worker deployed successfully!",
  "conversiondeployfailed": "Worker deployment failed",
  "conversionserversaved": "Server saved successfully",
  "conversionserverdeleted": "Server deleted successfully",
  "conversionloglevel": "Log Level",
  "conversionlogtype": "Log Type",
  "conversionlogtime": "Time",
  "conversionlogmessage": "Message",
  "conversionlogserver": "Server",
  "conversionlogtask": "Task",
  "conversionfilterall": "All",
  "conversionfilterconnection": "Connection",
  "conversionfiltertask": "Task",
  "conversionfilterhealth": "Health",
  "conversionfilterdeployment": "Deployment",
  "conversionfiltersystem": "System",
  "conversionnoservers": "No conversion servers configured. Click 'Add Server' to get started.",
  "conversionnologs": "No activity logs found for the selected filters.",
  "conversionlastheartbeat": "Last Heartbeat",
  "conversionjobscompleted": "Jobs Completed",
  "conversionjobsfailed": "Jobs Failed",
  "conversioncurrentload": "Current Load",
  "conversiondeploytitle": "Deploy Worker Files",
  "conversiondeployfiles": "Files to Deploy",
  "conversiondeployprogress": "Deployment Progress",
  "conversiondiagnosticstitle": "Server Diagnostics",
  "conversiondiagconnection": "Connection Status",
  "conversiondiagsystem": "System Information",
  "conversiondiagresources": "Resource Usage",
  "conversiondiagtasks": "Active Tasks",
  "conversiondiagerrors": "Recent Errors"
}

Related Documentation

• Video Upload Guide - Frontend video upload interface
• Video FTP Guide - Admin FTP bulk upload
• Video Processor Guide - Main video processing cron
• Storage Servers Guide - CDN/storage configuration

Version History

This guide is part of the ComusThumbz Admin Documentation. For the complete admin guide index, see Admin Guide.