Scorpi Wiki

You've loaded an old revision of the document! If you save it, you will create a new version with this data.

====== Discourse Installation on Fresh Ubuntu Server with Backup Restoration ======

===== Prerequisites =====
  * Fresh Ubuntu Server installation
  * Root or sudo access
  * Existing Discourse backup file (.tar.gz)
  * Domain name pointed to server IP

===== Installation Steps =====

==== 1. Initial System Setup ====
<code bash>
sudo -s
apt-get install git
</code>

==== 2. Clone Discourse Docker Repository ====
<code bash>
git clone https://github.com/discourse/discourse_docker.git /var/discourse
cd /var/discourse
chmod 700 containers
</code>

==== 3. Edit app.yml for Reverse Proxy Configuration ====

**Before running discourse-setup**, edit ''~/var/discourse/containers/app.yml'':

=== Comment Out SSL Templates ===
<code yaml>
templates:
  - "templates/postgres.template.yml"
  - "templates/redis.template.yml"
  - "templates/web.template.yml"
  - "templates/web.ratelimited.template.yml"
  ## SSL templates - COMMENTED OUT for reverse proxy setup
  # - "templates/web.ssl.template.yml"
  # - "templates/web.letsencrypt.ssl.template.yml"
</code>

=== Change Port Mapping ===
<code yaml>
expose:
  - "8080:80"   # expose container port 80 on host port 8080
  # - "443:443" # https - commented out
</code>

**Rationale:** Reverse proxy (nginx) will handle TLS termination and forward HTTP traffic to port 8080. Configuring this before setup avoids needing to rebuild the container later.

==== 4. Run Discourse Setup ====
<code bash>
./discourse-setup --skip-connection-test
</code>

**Note:** This process takes time. Good opportunity for a coffee break.

**Configuration Parameters:**

Provide these values during setup:

* **Hostname:** ''example.com''
  * **Developer Email:** ''admin@example.com''
  * **SMTP Server:** Your SMTP server address
  * **SMTP Port:** Typically ''587'' for TLS
  * **SMTP Username:** Your SMTP username
  * **SMTP Password:** Your SMTP password
  * **Notification Email:** Email address for outgoing notifications
  * **Let's Encrypt Email:** ''OFF'' (critical - disables Let's Encrypt since TLS will be handled by reverse proxy)

==== 5. Copy Backup File to Server ====

From your local machine (or source location), copy the backup file:
<code bash>
scp -P <ssh-port> backup-file.tar.gz user@example.com:/var/discourse/shared/standalone/backups/default/
</code>

**Notes:**
  * Replace ''<ssh-port>'' with your SSH port (default is 22, use ''-P 22'' or omit if default)
  * Replace ''backup-file.tar.gz'' with your actual backup filename
  * Adjust username and hostname as needed for your setup

==== 6. Restore Backup from Admin Panel ====

- Access Discourse at ''http://example.com'' (or server IP if DNS not yet configured)
  - Navigate to **Admin → Backups**
  - Locate the uploaded backup file in the list
  - Click **Restore** on the backup file
  - Confirm the restoration

**Note:** The restoration process will restart Discourse and may take several minutes depending on backup size.

==== 7. Install and Configure Nginx ====

#This nginx setup in this guide is but one valid configuration.
<code bash>
apt-get install nginx
</code>

==== 8. Create Nginx Configuration (Example) ====

**Note:** This configuration represents one working approach for proxying Discourse through nginx. Alternative configurations may work equally well depending on your specific requirements, existing infrastructure, or preferred nginx patterns.

Create ''/etc/nginx/sites-available/example.com'':
<code nginx>
server {
    listen 80;
    server_name example.com;
    
    location / {
        proxy_pass http://localhost:8080;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Forwarded-Host $host;
        
        # Required for Discourse
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        
        # Timeouts
        proxy_connect_timeout 60s;
        proxy_send_timeout 60s;
        proxy_read_timeout 60s;
        
        # Buffer settings
        proxy_buffering off;
        proxy_redirect off;
    }
    
    client_max_body_size 10m;
}
</code>

**Key Discourse Requirements:**
  * ''X-Forwarded-For'' and ''X-Forwarded-Host'' headers
  * WebSocket support (''proxy_http_version 1.1'', ''Upgrade'', ''Connection'' headers)
  * Buffering disabled (''proxy_buffering off'')
  * File upload size limit (''client_max_body_size 10m'')

==== 9. Enable Nginx Configuration ====
<code bash>
# Remove default site (prevents conflicts)
rm /etc/nginx/sites-enabled/default

# Create symlink to enable site
ln -s /etc/nginx/sites-available/example.com /etc/nginx/sites-enabled/

# Test configuration
nginx -t

# Reload nginx
systemctl reload nginx
</code>

==== 10. Install Certbot ====
<code bash>
snap install --classic certbot
</code>

==== 11. Generate SSL Certificate ====
<code bash>
certbot
</code>

Certbot will run interactively and:
  * Detect the nginx configuration for your domain
  * Prompt for domain selection
  * Automatically generate Let's Encrypt certificates
  * Modify nginx configuration to enable HTTPS
  * Configure automatic certificate renewal

==== 12. Reload Nginx ====
<code bash>
systemctl reload nginx
</code>

Your Discourse instance should now be accessible at ''https://example.com''

===== Important Notes =====

==== YAML Sensitivity ====
  * Be extremely careful with whitespace and alignment when editing ''app.yml''
  * Validate syntax at http://www.yamllint.com/ if needed

==== Undocumented Discourse Behavior ====
  * The ''8080:80'' port mapping for reverse proxy setups is not well-documented in official Discourse guides
  * Entering ''OFF'' for Let's Encrypt email is not clearly documented
  * Commenting out SSL templates is required but not obvious for reverse proxy scenarios
  * Discourse's Docker setup is optimized for their standard configuration, making reverse proxy setups unnecessarily opaque

==== Security Considerations ====
  * Use strong, unique passwords for SMTP and admin accounts
  * Secure transfer of backup files containing sensitive data
  * Review and update SMTP credentials appropriately for your environment
  * Consider firewall rules to restrict access to port 8080

===== Troubleshooting =====

==== Nginx Shows Default Welcome Page ====
  * Verify default site removed: ''ls /etc/nginx/sites-enabled/''
  * Confirm symlink exists: ''ls -la /etc/nginx/sites-enabled/example.com''
  * Test config: ''nginx -t''
  * Reload: ''systemctl reload nginx''

==== SSL Certificate Permission Errors ====
  * Run nginx test with sudo: ''sudo nginx -t''
  * Check for other configs referencing non-existent certificates
  * Ensure only HTTP (port 80) configuration exists before running certbot

==== Backup Not Visible in Admin Panel ====
  * Verify file copied to correct path: ''/var/discourse/shared/standalone/backups/default/''
  * Check file permissions: ''ls -la /var/discourse/shared/standalone/backups/default/''
  * Ensure filename ends in ''.tar.gz''

==== SCP Syntax Issues ====https://wiki.scorpi.us/doku.php?id=discourse&do=
  * Port flag is capital ''-P'' not lowercase ''-p''
  * Syntax: ''scp -P <port> <source> <user>@<host>:<destination>''
  * Example: ''scp -P 22 backup.tar.gz user@host:/path/''

```
===== Discourse API Configuration =====

==== Webhook Configuration ====

=== Available Events ===

**Topic Events**
  * Topic is created
  * Topic is revised
  * Topic is updated
  * Topic is deleted
  * Topic is recovered

**Post Events**
  * Post is created
  * Post is updated
  * Post is deleted
  * Post is recovered

**User Events**
  * User logged in
  * User logged out
  * User confirmed e-mail
  * User is created
  * User is approved
  * User is updated
  * User is deleted
  * User is suspended
  * User is unsuspended
  * User is anonymized

**Group Events**
  * Group is created
  * Group is updated
  * Group is deleted

**Category Events**
  * Category is created
  * Category is updated
  * Category is deleted

**Tag Events**
  * Tag is created
  * Tag is updated
  * Tag is deleted

**Chat Events** (Critical for sync)
  * Message is created
  * Message is edited
  * Message is trashed
  * Message is restored

**Other Events**
  * Reviewable Events
  * Notification Events
  * Solved Events
  * Badge Events
  * Group User Events
  * Like Events
  * User Promoted Events
  * Topic Voting Events

=== Webhook Settings ===

**Content Type:** ''application/json'' (recommended for easy parsing)

**Secret:** Optional string for generating HMAC signatures (implement signature verification to prevent spoofed webhooks)

**Filtering Options:**
  * Triggered Categories - only fire webhooks for specific categories
  * Triggered Tags - only fire webhooks for specific tags
  * Triggered Groups - only fire webhooks for specific groups
  * Leave blank to trigger for all

**TLS Certificate Check:** Enable unless using self-signed certs in development

=== Configured Webhook Endpoints ===

**hobbiesync webhook:**
  * URL: ''https://example.com/chat/hooks/WEBHOOK_ID_1''

**general webhook:**
  * URL: ''https://example.com/chat/hooks/WEBHOOK_ID_2''

Path structure: ''/chat/hooks/[uuid]'' suggests custom webhook handler service

==== Chat API Endpoints ====

Base path: ''/chat/api/channels''

=== Channel Operations ===

**List channels**
<code>
GET /chat/api/channels
</code>

**Get specific channel**
<code>
GET /chat/api/channels/{channel_id}
</code>

**List messages**
<code>
GET /chat/api/channels/{channel_id}/messages
</code>

**Send message**
<code>
POST /chat/api/channels/{channel_id}/messages
</code>

=== Authentication ===

Discourse API requires headers:
<code>
Api-Key: {your_api_key}
Api-Username: {bot_username}
</code>

**API Credentials:**
  * API Key: ''[REDACTED]''
  * Username: ''syncbot''

**Security Notes:**
  * Store credentials in environment variables or secrets management (Vault)
  * API key has full access as the bot user
  * Revoke/rotate if compromised
  * Ensure bot user has appropriate permissions (chat access, posting rights to target channels)

==== Architecture Considerations ====

=== Critical Events for Chat Sync ===

**Primary focus:**
  * Chat events (message created/edited/trashed/restored) - core sync functionality
  * Post events - if bridging forum posts to Discord channels
  * User events (login/logout/created) - for presence sync and user mapping

=== Bidirectional Sync Challenges ===

**Discourse → Discord:**
  * Webhooks handle this direction
  * Parse incoming webhook payloads
  * Map to Discord API calls

**Discord → Discourse:**
  * Discord bot with message event listeners
  * POST to Discourse API ''/chat/api/channels/{channel_id}/messages''
  * Requires Discord bot token and event subscriptions

**Loop Prevention:**
  * Track message IDs/sources to avoid infinite echo
  * Store mapping of Discourse message ID ↔ Discord message ID
  * Ignore messages from own bot user

=== State Management ===

Required mappings to store:
  * Message ID mapping (Discourse chat message ID ↔ Discord message ID)
  * User mapping (Discourse username ↔ Discord user/webhook representation)
  * Channel mapping (Discourse channel ID ↔ Discord channel ID)
  * Edit/delete operations require retrieving these mappings

=== Event Filtering Strategy ===

**Options:**
  * "Send me everything" - receives all events (high bandwidth/processing)
  * Selective events - choose specific event types
  * Category/tag/group filters - sync only specific Discourse areas to specific Discord channels

**Recommendation:** Use selective events and filtering to reduce noise and processing load

=== Implementation Flow ===

**Webhook Receiver (Discourse → Discord):**
  - Receive webhook POST request
  - Verify HMAC signature if secret configured
  - Parse JSON payload
  - Extract message/event data
  - Check for existing mapping (to detect edits/deletes)
  - Call Discord API
  - Store message ID mapping

**Discord Bot (Discord → Discourse):**
  - Listen for Discord message events
  - Check if message is from bot (skip to prevent loop)
  - Format message for Discourse
  - POST to Discourse API with credentials
  - Store message ID mapping

**Message History/Backfill:**
  * Use ''GET /chat/api/channels/{channel_id}/messages'' for initial sync
  * Implement pagination if needed
  * Consider rate limiting

Edit summary

Note: By editing this page you agree to license your content under the following license: GNU Free Documentation License 1.3