Table of Contents

Discourse Installation on Fresh Ubuntu Server with Backup Restoration

Prerequisites

Installation Steps

1. Initial System Setup

sudo -s
apt-get install git

2. Clone Discourse Docker Repository

git clone https://github.com/discourse/discourse_docker.git /var/discourse
cd /var/discourse
chmod 700 containers

3. Edit app.yml for Reverse Proxy Configuration

Before running discourse-setup, edit ~/var/discourse/containers/app.yml:

Comment Out SSL Templates

templates:
  - "templates/postgres.template.yml"
  - "templates/redis.template.yml"
  - "templates/web.template.yml"
  - "templates/web.ratelimited.template.yml"
  ## SSL templates - COMMENTED OUT for reverse proxy setup
  # - "templates/web.ssl.template.yml"
  # - "templates/web.letsencrypt.ssl.template.yml"

Change Port Mapping

expose:
  - "8080:80"   # expose container port 80 on host port 8080
  # - "443:443" # https - commented out

Rationale: Reverse proxy (nginx) will handle TLS termination and forward HTTP traffic to port 8080. Configuring this before setup avoids needing to rebuild the container later.

4. Run Discourse Setup

./discourse-setup --skip-connection-test

Note: This process takes time. Good opportunity for a coffee break.

Configuration Parameters:

Provide these values during setup:

5. Copy Backup File to Server

From your local machine (or source location), copy the backup file: 

scp -P <ssh-port> backup-file.tar.gz user@example.com:/var/discourse/shared/standalone/backups/default/

Notes:

6. Restore Backup from Admin Panel

  1. Access Discourse at http://example.com (or server IP if DNS not yet configured)
  2. Navigate to Admin → Backups
  3. Locate the uploaded backup file in the list
  4. Click Restore on the backup file
  5. Confirm the restoration

Note: The restoration process will restart Discourse and may take several minutes depending on backup size.

7. Install and Configure Nginx

#This nginx setup in this guide is but one valid configuration. 

apt-get install nginx

8. Create Nginx Configuration (Example)

Note: This configuration represents one working approach for proxying Discourse through nginx. Alternative configurations may work equally well depending on your specific requirements, existing infrastructure, or preferred nginx patterns.

Create /etc/nginx/sites-available/example.com: 

server {
    listen 80;
    server_name example.com;
 
    location / {
        proxy_pass http://localhost:8080;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Forwarded-Host $host;
 
        # Required for Discourse
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
 
        # Timeouts
        proxy_connect_timeout 60s;
        proxy_send_timeout 60s;
        proxy_read_timeout 60s;
 
        # Buffer settings
        proxy_buffering off;
        proxy_redirect off;
    }
 
    client_max_body_size 10m;
}

Key Discourse Requirements:

9. Enable Nginx Configuration

# Remove default site (prevents conflicts)
rm /etc/nginx/sites-enabled/default
 
# Create symlink to enable site
ln -s /etc/nginx/sites-available/example.com /etc/nginx/sites-enabled/
 
# Test configuration
nginx -t
 
# Reload nginx
systemctl reload nginx

10. Install Certbot

snap install --classic certbot

11. Generate SSL Certificate

certbot

Certbot will run interactively and:

12. Reload Nginx

systemctl reload nginx

Your Discourse instance should now be accessible at https://example.com

Important Notes

YAML Sensitivity

Undocumented Discourse Behavior

Security Considerations

Troubleshooting

Nginx Shows Default Welcome Page

SSL Certificate Permission Errors

Backup Not Visible in Admin Panel

==== SCP Syntax Issues ====https://wiki.scorpi.us/doku.php?id=discourse&do=

```

Discourse API Configuration

Webhook Configuration

Available Events

Topic Events

Post Events

User Events

Group Events

Category Events

Tag Events

Chat Events (Critical for sync)

Other Events

Webhook Settings

Content Type: application/json (recommended for easy parsing)

Secret: Optional string for generating HMAC signatures (implement signature verification to prevent spoofed webhooks)

Filtering Options:

TLS Certificate Check: Enable unless using self-signed certs in development

Configured Webhook Endpoints

hobbiesync webhook:

general webhook:

Path structure: /chat/hooks/[uuid] suggests custom webhook handler service

Chat API Endpoints

Base path: /chat/api/channels

Channel Operations

List channels 

GET /chat/api/channels

Get specific channel 

GET /chat/api/channels/{channel_id}

List messages 

GET /chat/api/channels/{channel_id}/messages

Send message 

POST /chat/api/channels/{channel_id}/messages

Authentication

Discourse API requires headers: 

Api-Key: {your_api_key}
Api-Username: {bot_username}

API Credentials:

Security Notes:

Architecture Considerations

Critical Events for Chat Sync

Primary focus:

Bidirectional Sync Challenges

Discourse → Discord:

Discord → Discourse:

Loop Prevention:

State Management

Required mappings to store:

Event Filtering Strategy

Options:

Recommendation: Use selective events and filtering to reduce noise and processing load

Implementation Flow

Webhook Receiver (Discourse → Discord):

  1. Receive webhook POST request
  2. Verify HMAC signature if secret configured
  3. Parse JSON payload
  4. Extract message/event data
  5. Check for existing mapping (to detect edits/deletes)
  6. Call Discord API
  7. Store message ID mapping

Discord Bot (Discord → Discourse):

  1. Listen for Discord message events
  2. Check if message is from bot (skip to prevent loop)
  3. Format message for Discourse
  4. POST to Discourse API with credentials
  5. Store message ID mapping

Message History/Backfill:

Profile Image/Avatar Synchronization

How Avatar Syncing Works

The bridge synchronizes profile images/avatars bidirectionally between Discord and Discourse using webhook-based user impersonation.

Discourse → Discord Avatar Flow

  1. Bridge polls Discourse Chat API every 1.5 seconds for new messages
  2. Each message includes user info with an avatar_template field
  3. Avatar template format: /user_avatar/site/username/{size}/123_2.png
  4. Bridge calls build_avatar_url() to construct full URL:
    • Replaces {size} placeholder with pixel size (default: 128)
    • Prepends Discourse base URL if path is relative
  5. Full avatar URL passed to Discord webhook avatar_url parameter
  6. Discord displays the message with the Discourse user's actual avatar
# Avatar URL construction (bridge.py lines 160-186)
def build_avatar_url(self, avatar_template: str, size: int = 128) -> Optional[str]:
    if not avatar_template:
        return None
    avatar_path = avatar_template.replace('{size}', str(size))
    if not avatar_path.startswith('/'):
        avatar_path = '/' + avatar_path
    return f"{self.base_url}{avatar_path}"

Discord → Discourse Avatar Flow

Discord user avatars are not synced to Discourse. Messages from Discord appear with the bot's configured username in Discourse, prefixed with [Discord] username: to identify the source user.

User Impersonation

The bridge uses Discord webhooks to display messages from Discourse users with their actual:

This creates a seamless experience where Discourse users appear as distinct users in Discord channels.

Image Upload/Download Synchronization

Discord → Discourse Images

  1. Discord attachments detected via message.attachments
  2. Images downloaded to temporary files using download_image()
  3. Uploaded to Discourse via /uploads.json?type=image endpoint
  4. Upload URL appended to message as markdown: ![]({image_url})
  5. Temporary files cleaned up after processing

Discourse → Discord Images

  1. Images extracted from uploads field in Discourse messages
  2. Relative URLs converted to absolute URLs (prepends base_url)
  3. URLs appended to message content
  4. Discord auto-embeds images from URLs

GIF and Video Handling

Giphy/Tenor URL Detection

The bridge detects and converts Giphy/Tenor share URLs to direct media URLs for proper embedding:

MP4 to GIF Conversion

When Giphy/Tenor returns MP4 URLs (common for larger animations), the bridge converts them to GIF using FFmpeg:

  1. Downloads MP4 to temporary file
  2. Generates optimized color palette for compression
  3. Converts to GIF with configurable settings:
    • Frame rate (default: 15 FPS)
    • Width scaling (default: 480px)
    • Duration limit (default: 10 seconds)
    • CPU thread limit (default: 3 threads)
  4. Uploads converted GIF to Discourse
  5. Cleans up temporary files

FFmpeg configuration options in bridge_config.json: 

{
  "bridge": {
    "ffmpeg_threads": 3,
    "ffmpeg_duration_limit": 10,
    "ffmpeg_fps": 15,
    "ffmpeg_scale_width": 480
  }
}

State Management

Message Tracking

The bridge tracks processed message IDs to prevent duplicates and enable resumption after restarts.

State file structure (bridge_state.json): 

{
  "discord_channels": {
    "channel_id": {
      "last_message_id": 1234567890,
      "last_updated": "2024-01-15T12:34:56Z"
    }
  },
  "discourse_channels": {
    "channel_id": {
      "last_message_id": 456,
      "last_updated": "2024-01-15T12:34:56Z"
    }
  }
}

State Initialization

On first run, the bridge initializes state with the latest message IDs to prevent flooding channels with historical messages.

Loop Prevention

The bridge implements multiple safeguards to prevent infinite message loops:

Discord → Discourse

Discourse → Discord

Configuration Reference

Full Configuration Structure

{
  "discord": {
    "token": "bot_token_here",
    "channels": [
      {
        "name": "channel_name",
        "discord_channel_id": 123456789012345678,
        "discourse_channel_id": 2,
        "discord_webhook_url": null,
        "discourse_webhook_url": "https://discourse.example.com/chat/hooks/UUID",
        "enabled": true
      }
    ]
  },
  "discourse": {
    "base_url": "https://discourse.example.com",
    "api_key": "api_key_here",
    "username": "syncbot"
  },
  "bridge": {
    "polling_interval": 1.5,
    "state_file": "bridge_state.json",
    "log_level": "INFO",
    "rate_limit_delay": 1.0,
    "use_discord_webhooks": true,
    "ffmpeg_threads": 3,
    "ffmpeg_duration_limit": 10,
    "ffmpeg_fps": 15,
    "ffmpeg_scale_width": 480
  }
}

Configuration Options

Option Default Description
polling_interval 1.5 Seconds between Discourse API polls
state_file bridge_state.json Path to state persistence file
log_level INFO Logging verbosity (DEBUG, INFO, WARNING, ERROR)
rate_limit_delay 1.0 Minimum seconds between API calls
use_discord_webhooks true Enable user impersonation via webhooks
ffmpeg_threads 3 CPU threads for MP4 conversion
ffmpeg_duration_limit 10 Max seconds of video to convert
ffmpeg_fps 15 Frame rate for converted GIFs
ffmpeg_scale_width 480 Width in pixels for converted GIFs

Environment Variable Overrides

Channel Mapping

Multi-Channel Support

The bridge supports multiple independent channel pairs. Each pair has:

Adding a New Channel Pair

  1. Add entry to discord.channels array in config
  2. Set discord_channel_id and discourse_channel_id
  3. Create Discourse webhook and add URL to discourse_webhook_url
  4. Set enabled: true
  5. Restart bridge - Discord webhook created automatically

Error Handling and Retry Logic

Automatic Retry

Graceful Degradation

Timeouts

Docker Deployment

Container Configuration

services:
  syncbot:
    build: .
    restart: unless-stopped
    deploy:
      resources:
        limits:
          cpus: '1'
          memory: 512M
    logging:
      driver: "json-file"
      options:
        max-size: "10m"
        max-file: "3"
    volumes:
      - ./bridge-config.json:/app/bridge-config.json:ro
      - ./bridge_state.json:/app/bridge_state.json

Security

Message Flow Diagrams

Discord → Discourse

Discord User posts message
         ↓
on_message() event triggered
         ↓
Check: Is webhook message? → Skip
Check: Is bot message? → Skip
         ↓
Find channel config by Discord channel ID
         ↓
Download image attachments (if any)
         ↓
Upload images to Discourse
         ↓
Extract/convert Giphy/Tenor URLs
         ↓
Convert MP4 to GIF (if needed)
         ↓
Format: "[Discord] username: message"
         ↓
Append image URLs as markdown
         ↓
Send via Discourse webhook
         ↓
Update Discord channel state

Discourse → Discord

poll_discourse() runs every 1.5 seconds
         ↓
For each enabled channel:
         ↓
GET messages after last_message_id
         ↓
For each new message:
         ↓
Check: Has [Discord] prefix? → Skip
         ↓
Extract username from user info
Extract avatar_template from user info
         ↓
Build avatar URL (replace {size}, add base_url)
         ↓
Extract image uploads, make URLs absolute
         ↓
Truncate content to 2000 chars if needed
         ↓
Send via Discord webhook:
  - username: Discourse username
  - avatar_url: Discourse avatar
  - content: message text
         ↓
Success? → Update Discourse channel state
Failed? → Retry on next poll

Dependencies

Python Packages

System Dependencies

Known Limitations

Not Currently Implemented

Troubleshooting

Messages Not Syncing

Duplicate Messages

Images Not Appearing

GIFs Not Converting