ComusThumbz 文档
HTML Markdown Print/PDF Plain Text
管理员登录

文档菜单


管理员访问

Gallery Bulk Import

Gallery Bulk Import - User Guide

Page Location: ct/admin/comus.add.php
Menu Path: Admin Panel → Galleries → Bulk Import
Access Level: Administrator (requires login)
Last Updated: 2026-01-01

Overview

The Gallery Bulk Import page allows administrators to import multiple gallery entries at once from CSV-formatted data. This powerful tool supports field mapping through customizable "playlists," model association, thumbnail downloading, duplicate detection, and asynchronous batch processing with real-time progress tracking.

[Screenshot: comus-add-main-form-overview]

Getting to This Page

  1. Log in to the admin panel at ct/admin/ctlogin.php
  2. In the left sidebar, click Galleries
  3. Click Bulk Import or navigate directly to ct/admin/comus.add.php
Note:
This page is designed for importing gallery data from external sources, TGP networks, or CSV exports. All data is first previewed before final import.

System Requirements

Configuration Required:
This page requires the following system dependencies to function properly.

PHP Requirements

Requirement Minimum Recommended Notes
PHP Version 7.4 8.2+ Uses null coalescing operators
memorylimit 256M 512M For large CSV files
maxexecutiontime 3000 3000 Set internally via iniset
defaultsockettimeout 3 3 Set internally for URL checks
maxinputvars 1000 5000 For large field mapping forms
postmaxsize 10M 50M For pasting large CSV data

Required PHP Extensions

Extension Required Purpose
mysqli Yes Database connectivity
session Yes CSRF token and session data storage
curl Yes URL validation, thumbnail downloading
json Yes Progress file handling, playlist storage
fileinfo Recommended MIME type detection for images
gd Recommended Thumbnail resize functionality

Required PHP Functions

Warning:
These functions must NOT be in disablefunctions in php.ini:
Function Required For Notes
fileputcontents() Progress tracking, logging Core functionality
filegetcontents() Reading progress files, CSV files Core functionality
rename() Atomic file writes For progress updates
curlinit() URL validation, thumbnail downloading Critical for imports
curlmultiinit() Parallel URL fetching Performance optimization
mkdir() Creating temp/logs directories First-time setup
touch() Creating log files First-time setup

Folder Permissions

Folder Path Permission Owner Purpose
ct/admin/temp/ 755 www-data Progress tracking files
ct/logs/ 755 www-data Import logging
ct/thumbs/ 755 www-data Downloaded thumbnails

Server Requirements

Component Requirement
Web Server Apache 2.4+ with modrewrite OR Nginx
Database MySQL 5.7+ / MariaDB 10.3+
cURL Required for URL validation

Folders Required

Folder Path Permission Created By Notes
ct/admin/temp/ 755 Script (auto-create) Progress tracking files
ct/logs/ 755 Install Script Import logging
ct/thumbs/ 755 Install Script Thumbnail storage

External Dependencies

Dependency Type Required Version Notes
Bootstrap 5.3 CDN Yes 5.3.0+ UI framework
Font Awesome 6 CDN Yes 6.5.1+ Icons

Page Layout

Header Section

The page header displays:

  • Title: "Gallery Bulk Import" with a photo icon
  • Breadcrumb: Dashboard > Galleries > Bulk Import
  • Description: "Import multiple galleries from CSV data with field mapping"

 

[Screenshot: comus-add-header]

Playlist Section

The playlist section allows you to save and load field mapping configurations:

Element Description
Load Existing Playlist Dropdown to select and auto-load saved configurations
Playlist Name Input field for naming new playlists
New Button Reset form for new playlist
Save Button Save current mappings to database
Delete Button Remove selected playlist

[Screenshot: comus-add-playlist-section]

 

Tip:
Save commonly used import configurations as playlists to speed up future imports from the same data source.

 

Field Mapping Section

The dynamic field mapping section lets you configure how CSV columns map to database fields:

Element Description
Column Number Sequential column identifier (1, 2, 3...)
Field Select Dropdown to choose which database field this column maps to
Field Description Contextual help showing field purpose and format
Add Column Add another column mapping (max 30)
Remove Column Remove a column mapping row

[Screenshot: comus-add-field-mapping]

Available Field Mappings

# Field Name Description
0 Url Full URL including http(s)://
1 Category Category name (created if not found)
2 Description Main description text
3 nPics Number of pictures in gallery
4 Vote Popularity rating 1-10 (1=most popular)
5 Thumb.jpg Thumbnail URL (system creates sized thumb)
6 keywords Space-separated keywords
7 hash System hash for deduplication
8 clicks Force initial click count
9 clickperiods Number of build cycles
10 rating Internal rating value
11 date Submit date (YYYYMMDDHHmm format)
12 status Acceptance status
13 clickpower Click power multiplier
14-17 Thm Src X/Y/W/H Master thumb source coordinates
18 Thm SrcUrl Master thumbnail source URL
19 has video Video flag (0/1)
20 Explicit Explicit content flag (0/1)
21 error404 404 error flag (0/1)
22 Reviewed Review status flag (0/1)
23 Sponsor Code Sponsor identifier
24 Title Gallery title
25 Desc 2 Secondary description
26 Path As File Treat path as file flag (0/1)
27 Alt Thumb Alternate thumbnail URL
28 n/a Ignore this column
29 Model Model name(s), comma separated

Import Data Section

Element Description
Gallery Data Textarea Paste CSV data, one entry per line
Delimiter Field separator character (default: comma)
CSV File Path Alternative: server path to CSV file
Thumb Path URL prefix to prepend to thumbnail column values

Import Options

Option Description
Fast scan Skip URL validation checks for faster import
Check dupes Reject entries matching existing URL or hash
Within category Check duplicates only within same category
Copy masters too Download master thumbnail in addition to sized thumb
Resize thumbs Use deferred thumbnail resize workflow
Setting Description
Accept Set status: Approved, Pending, or Don't change
Force Category Override category for all entries
Force Model Override model assignment for all entries

[Screenshot: comus-add-import-options]

Features & Functions

Field Mapping Playlists

Purpose: Save and reuse field mapping configurations for repeated imports from the same source.

How to use:

  1. Configure your field mappings using the column dropdowns
  2. Enter a unique name in the "Playlist Name" field
  3. Click Save to store the configuration
  4. Future imports can load this configuration instantly via the dropdown

Playlist data stored:

  • Field mappings (which column = which field)
  • Delimiter setting
  • All import options (force category, force model, etc.)
Tip:
Create separate playlists for different data sources: "TGP Network A", "Manual CSV Export", "CMS Export", etc.

Duplicate Detection

Purpose: Prevent duplicate gallery entries from being imported.

Modes:

Mode Behavior
Check dupes (global) Reject if URL or hash exists anywhere in database
Within category Reject only if duplicate exists in same category
Neither checked Allow all entries regardless of duplicates

Detection method: Compares URL and MD5 hash values against existing tblTgp entries.

Model Association

Purpose: Link galleries to performer models during import.

How it works:

  1. Map a CSV column to field #29 (Model)
  2. Enter model names, comma-separated
  3. During import, each name is:
  • Looked up in tblModels
  • Created if not found (with Active status)
  • Linked via tblGalleryModels join table

Force Model option: Overrides the Model column with a single model for all entries.

URL Validation (Non-Fast Scan)

When Fast scan is NOT enabled, each gallery URL is validated:

  1. URL format is checked and fixed
  2. HTTP request is sent with cURL
  3. Response code is verified (must be < 400)
  4. Page content is scanned for video file extensions
  5. Invalid URLs are rejected with error logging
Warning:
URL validation significantly increases import time. Use Fast scan for trusted data sources.

Thumbnail Download

For each imported gallery with a thumbnail URL:

Thumbnail is downloaded via cURL

  1. Saved to ct/thumbs/tn{galleryid}.jpg
  2. Image dimensions are detected with getimagesize()
  3. Database record is updated with size/dimensions
  4. Optional FTP distribution if configured

Asynchronous Processing

The import runs asynchronously with real-time progress tracking:

  1. Preview Step: Data is validated and temp files created
  2. Run Step: AJAX call triggers background processing
  3. Status Polling: Frontend polls for progress every 500ms
  4. Progress File: JSON file tracks done/total/logs

Progress tracking files:

  • ct/admin/temp/galleryimportprogress.json - Current status
  • ct/admin/temp/galleryimportdata - Raw import data
  • ct/admin/temp/galleryimportxlat - Field mappings (serialized)
  • ct/admin/temp/galleryimportopts.json - Import options

 

Step-by-Step Usage

Task 1: Basic CSV Import

Note:
This workflow imports galleries from simple CSV data with URL, category, and description.

Steps:

  1. Navigate to Admin Panel → Galleries → Bulk Import
  1. Configure field mappings:
  • Column 1: Select "Url"
  • Column 2: Select "Category"
  • Column 3: Select "Description"
  • Click Add Column for additional fields
  1. Paste your CSV data in the textarea:
http://example.com/gallery1,Amateurs,Hot amateur gallery
   http://example.com/gallery2,Brunettes,Stunning brunette photos
  1. Verify delimiter is set correctly (comma, tab, etc.)
  1. Set Accept to "Approved" or "Pending"
  1. Check Fast scan for quick import
  1. Click Preview Import
  1. Review the preview table showing parsed data
  1. Click Confirm & Start Import
  1. Monitor real-time progress bar and log messages

Result: Galleries are added to tblTgp with specified status.

[Screenshot: comus-add-progress-screen]

Task 2: Import with Model Association

Steps:

  1. Add a column mapping for field #29 (Model)
  1. Include model names in your CSV:
http://example.com/gallery1,Amateurs,Description,Jane Doe,Anna Smith
  1. Models in the Model column can be comma-separated
  1. Run the import

Result: Galleries are created AND linked to models via tblGalleryModels.

Task 3: Using Saved Playlists

Steps:

  1. Configure all field mappings and options
  1. Enter a playlist name (e.g., "TGP Network A")
  1. Click Save
  1. For future imports, select the playlist from the dropdown
  1. Settings are automatically loaded
  1. Paste new data and proceed with import

Progress Interface

During import, the progress interface displays:

Progress Bar

  • Animated striped bar showing percentage complete
  • Percentage text (e.g., "45%")

Status Line

  • Shows "Processed X of Y" count
  • Indicates "Complete" when finished

Log Panel

  • Scrollable list of import events
  • Color-coded messages:
  • Green: Success messages
  • Yellow: Warnings
  • Red: Errors

Statistics Cards

Card Description
Galleries Added Total successfully imported
Thumbs Saved Thumbnails downloaded successfully
Thumb Failures Failed thumbnail downloads

Navigation Buttons

  • Admin Home: Return to dashboard
  • New Import: Start a fresh import

[Screenshot: comus-add-progress-stats]

Error Messages & Troubleshooting

Common Errors

Error Message Cause Solution
"Import data not found" Preview files missing Go back and run Preview step again
"Cannot create temp dir" Permission issue Ensure ct/admin/temp/ is writable (755)
"Duplicate within 'Category'" Dupe detection triggered Entry exists; disable dupe check or skip
"Invalid URL on line X" Malformed URL Check URL format, must start with http(s)://
"Bad URL: HTTP 404" Gallery page not found URL is broken; source issue
"DB error" Database insert failed Check field lengths, special characters
"Playlist already exists" Duplicate playlist name Choose a different name

Troubleshooting

Problem: Import stalls at 0%
Solution:

  • Check browser console for JavaScript errors
  • Verify ct/admin/temp/ folder exists and is writable
  • Check ct/logs/galleryadd.log for PHP errors

Problem: Thumbnails not saving
Solution:

  • Verify ct/thumbs/ folder exists with write permissions
  • Check if thumbnail URLs are accessible
  • Review log for cURL errors

Problem: Models not linking
Solution:

  • Verify column is mapped to field #29 (Model)
  • Check model names match exactly (case-sensitive)
  • Review log for model creation messages

Problem: Progress bar shows but logs are empty
Solution:

  • Session may have expired; re-login
  • Progress JSON file may be corrupted; delete temp files and retry
  • Gallery Review (ct/admin/admin.php) - Manage imported galleries
  • Models (ct/admin/models.php) - Manage model records
  • Categories (ct/admin/categories.php) - Manage category list

Tips & Best Practices

Tip:
Use Fast Scan for trusted sources: If importing from a known-good source, enable Fast scan to skip URL validation and dramatically speed up imports.
Tip:
Create playlists for each source: Save time by creating dedicated playlists for each data source you regularly import from.
Tip:
Test with small batches first: Before importing thousands of entries, test with 5-10 rows to verify field mappings are correct.
Warning:
Avoid special characters in descriptions: Commas, quotes, and backslashes in description fields can break CSV parsing. Use tab delimiter or clean data first.
Tip:
Check logs after import: Always review ct/admin/log_viewer.php after large imports to identify any issues.

Changelog

Date Version Changes
2026-01-01 1.0 Initial guide created

Related Pages