Table of Contents
Discourse Installation on Fresh Ubuntu Server with Backup Restoration
Prerequisites
- Fresh Ubuntu Server installation
- Root or sudo access
- Existing Discourse backup file (.tar.gz)
- Domain name pointed to server IP
Installation Steps
1. Initial System Setup
sudo -s apt-get install git
2. Clone Discourse Docker Repository
git clone https://github.com/discourse/discourse_docker.git /var/discourse cd /var/discourse chmod 700 containers
3. Edit app.yml for Reverse Proxy Configuration
Before running discourse-setup, edit ~/var/discourse/containers/app.yml:
Comment Out SSL Templates
templates: - "templates/postgres.template.yml" - "templates/redis.template.yml" - "templates/web.template.yml" - "templates/web.ratelimited.template.yml" ## SSL templates - COMMENTED OUT for reverse proxy setup # - "templates/web.ssl.template.yml" # - "templates/web.letsencrypt.ssl.template.yml"
Change Port Mapping
expose: - "8080:80" # expose container port 80 on host port 8080 # - "443:443" # https - commented out
Rationale: Reverse proxy (nginx) will handle TLS termination and forward HTTP traffic to port 8080. Configuring this before setup avoids needing to rebuild the container later.
4. Run Discourse Setup
./discourse-setup --skip-connection-test
Note: This process takes time. Good opportunity for a coffee break.
Configuration Parameters:
Provide these values during setup:
- Hostname:
example.com - Developer Email:
admin@example.com - SMTP Server: Your SMTP server address
- SMTP Port: Typically
587for TLS - SMTP Username: Your SMTP username
- SMTP Password: Your SMTP password
- Notification Email: Email address for outgoing notifications
- Let's Encrypt Email:
OFF(critical - disables Let's Encrypt since TLS will be handled by reverse proxy)
5. Copy Backup File to Server
From your local machine (or source location), copy the backup file:
scp -P <ssh-port> backup-file.tar.gz user@example.com:/var/discourse/shared/standalone/backups/default/
Notes:
- Replace
<ssh-port>with your SSH port (default is 22, use-P 22or omit if default) - Replace
backup-file.tar.gzwith your actual backup filename - Adjust username and hostname as needed for your setup
6. Restore Backup from Admin Panel
- Access Discourse at
http://example.com(or server IP if DNS not yet configured) - Navigate to Admin → Backups
- Locate the uploaded backup file in the list
- Click Restore on the backup file
- Confirm the restoration
Note: The restoration process will restart Discourse and may take several minutes depending on backup size.
7. Install and Configure Nginx
#This nginx setup in this guide is but one valid configuration.
apt-get install nginx
8. Create Nginx Configuration (Example)
Note: This configuration represents one working approach for proxying Discourse through nginx. Alternative configurations may work equally well depending on your specific requirements, existing infrastructure, or preferred nginx patterns.
Create /etc/nginx/sites-available/example.com:
server { listen 80; server_name example.com; location / { proxy_pass http://localhost:8080; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Host $host; # Required for Discourse proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "upgrade"; # Timeouts proxy_connect_timeout 60s; proxy_send_timeout 60s; proxy_read_timeout 60s; # Buffer settings proxy_buffering off; proxy_redirect off; } client_max_body_size 10m; }
Key Discourse Requirements:
X-Forwarded-ForandX-Forwarded-Hostheaders- WebSocket support (
proxy_http_version 1.1,Upgrade,Connectionheaders) - Buffering disabled (
proxy_buffering off) - File upload size limit (
client_max_body_size 10m)
9. Enable Nginx Configuration
# Remove default site (prevents conflicts) rm /etc/nginx/sites-enabled/default # Create symlink to enable site ln -s /etc/nginx/sites-available/example.com /etc/nginx/sites-enabled/ # Test configuration nginx -t # Reload nginx systemctl reload nginx
10. Install Certbot
snap install --classic certbot
11. Generate SSL Certificate
certbot
Certbot will run interactively and:
- Detect the nginx configuration for your domain
- Prompt for domain selection
- Automatically generate Let's Encrypt certificates
- Modify nginx configuration to enable HTTPS
- Configure automatic certificate renewal
12. Reload Nginx
systemctl reload nginx
Your Discourse instance should now be accessible at https://example.com
Important Notes
YAML Sensitivity
- Be extremely careful with whitespace and alignment when editing
app.yml - Validate syntax at http://www.yamllint.com/ if needed
Undocumented Discourse Behavior
- The
8080:80port mapping for reverse proxy setups is not well-documented in official Discourse guides - Entering
OFFfor Let's Encrypt email is not clearly documented - Commenting out SSL templates is required but not obvious for reverse proxy scenarios
- Discourse's Docker setup is optimized for their standard configuration, making reverse proxy setups unnecessarily opaque
Security Considerations
- Use strong, unique passwords for SMTP and admin accounts
- Secure transfer of backup files containing sensitive data
- Review and update SMTP credentials appropriately for your environment
- Consider firewall rules to restrict access to port 8080
Troubleshooting
Nginx Shows Default Welcome Page
- Verify default site removed:
ls /etc/nginx/sites-enabled/ - Confirm symlink exists:
ls -la /etc/nginx/sites-enabled/example.com - Test config:
nginx -t - Reload:
systemctl reload nginx
SSL Certificate Permission Errors
- Run nginx test with sudo:
sudo nginx -t - Check for other configs referencing non-existent certificates
- Ensure only HTTP (port 80) configuration exists before running certbot
Backup Not Visible in Admin Panel
- Verify file copied to correct path:
/var/discourse/shared/standalone/backups/default/ - Check file permissions:
ls -la /var/discourse/shared/standalone/backups/default/ - Ensure filename ends in
.tar.gz
==== SCP Syntax Issues ====https://wiki.scorpi.us/doku.php?id=discourse&do=
- Port flag is capital
-Pnot lowercase-p - Syntax:
scp -P <port> <source> <user>@<host>:<destination> - Example:
scp -P 22 backup.tar.gz user@host:/path/
```
Discourse API Configuration
Webhook Configuration
Available Events
Topic Events
- Topic is created
- Topic is revised
- Topic is updated
- Topic is deleted
- Topic is recovered
Post Events
- Post is created
- Post is updated
- Post is deleted
- Post is recovered
User Events
- User logged in
- User logged out
- User confirmed e-mail
- User is created
- User is approved
- User is updated
- User is deleted
- User is suspended
- User is unsuspended
- User is anonymized
Group Events
- Group is created
- Group is updated
- Group is deleted
Category Events
- Category is created
- Category is updated
- Category is deleted
Tag Events
- Tag is created
- Tag is updated
- Tag is deleted
Chat Events (Critical for sync)
- Message is created
- Message is edited
- Message is trashed
- Message is restored
Other Events
- Reviewable Events
- Notification Events
- Solved Events
- Badge Events
- Group User Events
- Like Events
- User Promoted Events
- Topic Voting Events
Webhook Settings
Content Type: application/json (recommended for easy parsing)
Secret: Optional string for generating HMAC signatures (implement signature verification to prevent spoofed webhooks)
Filtering Options:
- Triggered Categories - only fire webhooks for specific categories
- Triggered Tags - only fire webhooks for specific tags
- Triggered Groups - only fire webhooks for specific groups
- Leave blank to trigger for all
TLS Certificate Check: Enable unless using self-signed certs in development
Configured Webhook Endpoints
hobbiesync webhook:
general webhook:
Path structure: /chat/hooks/[uuid] suggests custom webhook handler service
Chat API Endpoints
Base path: /chat/api/channels
Channel Operations
List channels
GET /chat/api/channels
Get specific channel
GET /chat/api/channels/{channel_id}
List messages
GET /chat/api/channels/{channel_id}/messages
Send message
POST /chat/api/channels/{channel_id}/messages
Authentication
Discourse API requires headers:
Api-Key: {your_api_key}
Api-Username: {bot_username}
API Credentials:
- API Key:
[REDACTED] - Username:
syncbot
Security Notes:
- Store credentials in environment variables or secrets management (Vault)
- API key has full access as the bot user
- Revoke/rotate if compromised
- Ensure bot user has appropriate permissions (chat access, posting rights to target channels)
Architecture Considerations
Critical Events for Chat Sync
Primary focus:
- Chat events (message created/edited/trashed/restored) - core sync functionality
- Post events - if bridging forum posts to Discord channels
- User events (login/logout/created) - for presence sync and user mapping
Bidirectional Sync Challenges
Discourse → Discord:
- Webhooks handle this direction
- Parse incoming webhook payloads
- Map to Discord API calls
Discord → Discourse:
- Discord bot with message event listeners
- POST to Discourse API
/chat/api/channels/{channel_id}/messages - Requires Discord bot token and event subscriptions
Loop Prevention:
- Track message IDs/sources to avoid infinite echo
- Store mapping of Discourse message ID ↔ Discord message ID
- Ignore messages from own bot user
State Management
Required mappings to store:
- Message ID mapping (Discourse chat message ID ↔ Discord message ID)
- User mapping (Discourse username ↔ Discord user/webhook representation)
- Channel mapping (Discourse channel ID ↔ Discord channel ID)
- Edit/delete operations require retrieving these mappings
Event Filtering Strategy
Options:
- “Send me everything” - receives all events (high bandwidth/processing)
- Selective events - choose specific event types
- Category/tag/group filters - sync only specific Discourse areas to specific Discord channels
Recommendation: Use selective events and filtering to reduce noise and processing load
Implementation Flow
Webhook Receiver (Discourse → Discord):
- Receive webhook POST request
- Verify HMAC signature if secret configured
- Parse JSON payload
- Extract message/event data
- Check for existing mapping (to detect edits/deletes)
- Call Discord API
- Store message ID mapping
Discord Bot (Discord → Discourse):
- Listen for Discord message events
- Check if message is from bot (skip to prevent loop)
- Format message for Discourse
- POST to Discourse API with credentials
- Store message ID mapping
Message History/Backfill:
- Use
GET /chat/api/channels/{channel_id}/messagesfor initial sync - Implement pagination if needed
- Consider rate limiting
Profile Image/Avatar Synchronization
How Avatar Syncing Works
The bridge synchronizes profile images/avatars bidirectionally between Discord and Discourse using webhook-based user impersonation.
Discourse → Discord Avatar Flow
- Bridge polls Discourse Chat API every 1.5 seconds for new messages
- Each message includes user info with an
avatar_templatefield - Avatar template format:
/user_avatar/site/username/{size}/123_2.png - Bridge calls
build_avatar_url()to construct full URL:- Replaces
{size}placeholder with pixel size (default: 128) - Prepends Discourse base URL if path is relative
- Full avatar URL passed to Discord webhook
avatar_urlparameter - Discord displays the message with the Discourse user's actual avatar
# Avatar URL construction (bridge.py lines 160-186) def build_avatar_url(self, avatar_template: str, size: int = 128) -> Optional[str]: if not avatar_template: return None avatar_path = avatar_template.replace('{size}', str(size)) if not avatar_path.startswith('/'): avatar_path = '/' + avatar_path return f"{self.base_url}{avatar_path}"
Discord → Discourse Avatar Flow
Discord user avatars are not synced to Discourse. Messages from Discord appear with the bot's configured username in Discourse, prefixed with [Discord] username: to identify the source user.
User Impersonation
The bridge uses Discord webhooks to display messages from Discourse users with their actual:
- Username - Discourse display name shown in Discord
- Avatar - Discourse profile image displayed in Discord
This creates a seamless experience where Discourse users appear as distinct users in Discord channels.
Image Upload/Download Synchronization
Discord → Discourse Images
- Discord attachments detected via
message.attachments - Images downloaded to temporary files using
download_image() - Uploaded to Discourse via
/uploads.json?type=imageendpoint - Upload URL appended to message as markdown:
 - Temporary files cleaned up after processing
Discourse → Discord Images
- Images extracted from
uploadsfield in Discourse messages - Relative URLs converted to absolute URLs (prepends base_url)
- URLs appended to message content
- Discord auto-embeds images from URLs
GIF and Video Handling
Giphy/Tenor URL Detection
The bridge detects and converts Giphy/Tenor share URLs to direct media URLs for proper embedding:
- Giphy:
https://giphy.com/gifs/…→https://media.giphy.com/media/{ID}/giphy.gif - Tenor:
https://tenor.com/view/…→https://media.tenor.com/{ID}/tenor.gif
MP4 to GIF Conversion
When Giphy/Tenor returns MP4 URLs (common for larger animations), the bridge converts them to GIF using FFmpeg:
- Downloads MP4 to temporary file
- Generates optimized color palette for compression
- Converts to GIF with configurable settings:
- Frame rate (default: 15 FPS)
- Width scaling (default: 480px)
- Duration limit (default: 10 seconds)
- CPU thread limit (default: 3 threads)
- Uploads converted GIF to Discourse
- Cleans up temporary files
FFmpeg configuration options in bridge_config.json:
{
"bridge": {
"ffmpeg_threads": 3,
"ffmpeg_duration_limit": 10,
"ffmpeg_fps": 15,
"ffmpeg_scale_width": 480
}
}
State Management
Message Tracking
The bridge tracks processed message IDs to prevent duplicates and enable resumption after restarts.
State file structure (bridge_state.json):
{
"discord_channels": {
"channel_id": {
"last_message_id": 1234567890,
"last_updated": "2024-01-15T12:34:56Z"
}
},
"discourse_channels": {
"channel_id": {
"last_message_id": 456,
"last_updated": "2024-01-15T12:34:56Z"
}
}
}
State Initialization
On first run, the bridge initializes state with the latest message IDs to prevent flooding channels with historical messages.
Loop Prevention
The bridge implements multiple safeguards to prevent infinite message loops:
Discord → Discourse
- Messages prefixed with
[Discord]tag - Webhook messages (non-user) are detected and skipped
- Bot's own messages filtered by user ID
Discourse → Discord
- Messages containing
[Discord]prefix are ignored - Prevents echoing messages that originated from Discord
Configuration Reference
Full Configuration Structure
{
"discord": {
"token": "bot_token_here",
"channels": [
{
"name": "channel_name",
"discord_channel_id": 123456789012345678,
"discourse_channel_id": 2,
"discord_webhook_url": null,
"discourse_webhook_url": "https://discourse.example.com/chat/hooks/UUID",
"enabled": true
}
]
},
"discourse": {
"base_url": "https://discourse.example.com",
"api_key": "api_key_here",
"username": "syncbot"
},
"bridge": {
"polling_interval": 1.5,
"state_file": "bridge_state.json",
"log_level": "INFO",
"rate_limit_delay": 1.0,
"use_discord_webhooks": true,
"ffmpeg_threads": 3,
"ffmpeg_duration_limit": 10,
"ffmpeg_fps": 15,
"ffmpeg_scale_width": 480
}
}
Configuration Options
| Option | Default | Description |
|---|---|---|
polling_interval | 1.5 | Seconds between Discourse API polls |
state_file | bridge_state.json | Path to state persistence file |
log_level | INFO | Logging verbosity (DEBUG, INFO, WARNING, ERROR) |
rate_limit_delay | 1.0 | Minimum seconds between API calls |
use_discord_webhooks | true | Enable user impersonation via webhooks |
ffmpeg_threads | 3 | CPU threads for MP4 conversion |
ffmpeg_duration_limit | 10 | Max seconds of video to convert |
ffmpeg_fps | 15 | Frame rate for converted GIFs |
ffmpeg_scale_width | 480 | Width in pixels for converted GIFs |
Environment Variable Overrides
DISCORD_TOKEN- Discord bot tokenDISCOURSE_API_KEY- Discourse API keyDISCOURSE_USERNAME- Discourse bot usernameDISCOURSE_BASE_URL- Discourse instance URLLOG_LEVEL- Logging levelCONFIG_FILE- Path to config fileSTATE_FILE- Path to state file
Channel Mapping
Multi-Channel Support
The bridge supports multiple independent channel pairs. Each pair has:
- Discord channel ID
- Discourse channel ID
- Separate webhook URLs
- Independent enable/disable flag
- Separate state tracking
Adding a New Channel Pair
- Add entry to
discord.channelsarray in config - Set
discord_channel_idanddiscourse_channel_id - Create Discourse webhook and add URL to
discourse_webhook_url - Set
enabled: true - Restart bridge - Discord webhook created automatically
Error Handling and Retry Logic
Automatic Retry
- Failed Discord webhook sends do not update state
- Message remains in queue and retries on next poll cycle
- Prevents message loss on temporary failures
Graceful Degradation
- API errors caught and logged, processing continues
- Image download failures don't block message delivery
- FFmpeg conversion failures fall back to posting MP4 URL
Timeouts
- API requests: 30-60 second timeout
- FFmpeg conversion: 30-60 second timeout
- Prevents indefinite hangs
Docker Deployment
Container Configuration
services: syncbot: build: . restart: unless-stopped deploy: resources: limits: cpus: '1' memory: 512M logging: driver: "json-file" options: max-size: "10m" max-file: "3" volumes: - ./bridge-config.json:/app/bridge-config.json:ro - ./bridge_state.json:/app/bridge_state.json
Security
- Runs as non-root user inside container
- Config mounted read-only
- State file writable for persistence
- Resource limits prevent runaway processes
Message Flow Diagrams
Discord → Discourse
Discord User posts message
↓
on_message() event triggered
↓
Check: Is webhook message? → Skip
Check: Is bot message? → Skip
↓
Find channel config by Discord channel ID
↓
Download image attachments (if any)
↓
Upload images to Discourse
↓
Extract/convert Giphy/Tenor URLs
↓
Convert MP4 to GIF (if needed)
↓
Format: "[Discord] username: message"
↓
Append image URLs as markdown
↓
Send via Discourse webhook
↓
Update Discord channel state
Discourse → Discord
poll_discourse() runs every 1.5 seconds
↓
For each enabled channel:
↓
GET messages after last_message_id
↓
For each new message:
↓
Check: Has [Discord] prefix? → Skip
↓
Extract username from user info
Extract avatar_template from user info
↓
Build avatar URL (replace {size}, add base_url)
↓
Extract image uploads, make URLs absolute
↓
Truncate content to 2000 chars if needed
↓
Send via Discord webhook:
- username: Discourse username
- avatar_url: Discourse avatar
- content: message text
↓
Success? → Update Discourse channel state
Failed? → Retry on next poll
Dependencies
Python Packages
discord.py>=2.0.0- Discord bot frameworkrequests>=2.28.0- HTTP requestsaiohttp>=3.8.0- Async HTTPpython-dotenv>=0.19.0- Environment variables
System Dependencies
- Python 3.11+
- FFmpeg (for MP4 to GIF conversion)
Known Limitations
Not Currently Implemented
- Message Edit Sync - Edits are not synchronized
- Message Delete Sync - Deletions are not synchronized
- Reaction Sync - Emoji reactions not bridged
- PluralKit Support - Removed in current version (was in earlier versions per git history)
- Discord Avatar → Discourse - Only Discourse avatars sync to Discord, not vice versa
Troubleshooting
Messages Not Syncing
- Check
enabled: truein channel config - Verify Discord bot has webhook permissions in channel
- Check Discourse webhook URL is correct
- Review logs for API errors
Duplicate Messages
- Check state file permissions (must be writable)
- Verify state file path matches config
Images Not Appearing
- Verify Discourse API key has upload permissions
- Check temporary directory is writable
- Review logs for upload errors
GIFs Not Converting
- Ensure FFmpeg is installed:
ffmpeg -version - Check FFmpeg is in PATH
- Review logs for conversion errors
- Try reducing
ffmpeg_duration_limitorffmpeg_scale_width
