Differences

This shows you the differences between two versions of the page.

--- discourse [2026/01/23 06:05] – 208.92.105.181
+++ discourse [2026/01/26 21:07] (current) – 208.92.105.160
@@ Line 90: / Line 90: @@
 ==== 7. Install and Configure Nginx ====
+#This nginx setup in this guide is but one valid configuration.
 <code bash>
 apt-get install nginx
@@ Line 216: / Line 218: @@
   * Syntax: ''scp -P <port> <source> <user>@<host>:<destination>''
   * Example: ''scp -P 22 backup.tar.gz user@host:/path/''
+```
+===== Discourse API Configuration =====
+==== Webhook Configuration ====
+=== Available Events ===
+**Topic Events**
+  * Topic is created
+  * Topic is revised
+  * Topic is updated
+  * Topic is deleted
+  * Topic is recovered
+**Post Events**
+  * Post is created
+  * Post is updated
+  * Post is deleted
+  * Post is recovered
+**User Events**
+  * User logged in
+  * User logged out
+  * User confirmed e-mail
+  * User is created
+  * User is approved
+  * User is updated
+  * User is deleted
+  * User is suspended
+  * User is unsuspended
+  * User is anonymized
+**Group Events**
+  * Group is created
+  * Group is updated
+  * Group is deleted
+**Category Events**
+  * Category is created
+  * Category is updated
+  * Category is deleted
+**Tag Events**
+  * Tag is created
+  * Tag is updated
+  * Tag is deleted
+**Chat Events** (Critical for sync)
+  * Message is created
+  * Message is edited
+  * Message is trashed
+  * Message is restored
+**Other Events**
+  * Reviewable Events
+  * Notification Events
+  * Solved Events
+  * Badge Events
+  * Group User Events
+  * Like Events
+  * User Promoted Events
+  * Topic Voting Events
+=== Webhook Settings ===
+**Content Type:** ''application/json'' (recommended for easy parsing)
+**Secret:** Optional string for generating HMAC signatures (implement signature verification to prevent spoofed webhooks)
+**Filtering Options:**
+  * Triggered Categories - only fire webhooks for specific categories
+  * Triggered Tags - only fire webhooks for specific tags
+  * Triggered Groups - only fire webhooks for specific groups
+  * Leave blank to trigger for all
+**TLS Certificate Check:** Enable unless using self-signed certs in development
+=== Configured Webhook Endpoints ===
+**hobbiesync webhook:**
+  * URL: ''https://example.com/chat/hooks/WEBHOOK_ID_1''
+**general webhook:**
+  * URL: ''https://example.com/chat/hooks/WEBHOOK_ID_2''
+Path structure: ''/chat/hooks/[uuid]'' suggests custom webhook handler service
+==== Chat API Endpoints ====
+Base path: ''/chat/api/channels''
+=== Channel Operations ===
+**List channels**
+<code>
+GET /chat/api/channels
+</code>
+**Get specific channel**
+<code>
+GET /chat/api/channels/{channel_id}
+</code>
+**List messages**
+<code>
+GET /chat/api/channels/{channel_id}/messages
+</code>
+**Send message**
+<code>
+POST /chat/api/channels/{channel_id}/messages
+</code>
+=== Authentication ===
+Discourse API requires headers:
+<code>
+Api-Key: {your_api_key}
+Api-Username: {bot_username}
+</code>
+**API Credentials:**
+  * API Key: ''[REDACTED]''
+  * Username: ''syncbot''
+**Security Notes:**
+  * Store credentials in environment variables or secrets management (Vault)
+  * API key has full access as the bot user
+  * Revoke/rotate if compromised
+  * Ensure bot user has appropriate permissions (chat access, posting rights to target channels)
+==== Architecture Considerations ====
+=== Critical Events for Chat Sync ===
+**Primary focus:**
+  * Chat events (message created/edited/trashed/restored) - core sync functionality
+  * Post events - if bridging forum posts to Discord channels
+  * User events (login/logout/created) - for presence sync and user mapping
+=== Bidirectional Sync Challenges ===
+**Discourse → Discord:**
+  * Webhooks handle this direction
+  * Parse incoming webhook payloads
+  * Map to Discord API calls
+**Discord → Discourse:**
+  * Discord bot with message event listeners
+  * POST to Discourse API ''/chat/api/channels/{channel_id}/messages''
+  * Requires Discord bot token and event subscriptions
+**Loop Prevention:**
+  * Track message IDs/sources to avoid infinite echo
+  * Store mapping of Discourse message ID ↔ Discord message ID
+  * Ignore messages from own bot user
+=== State Management ===
+Required mappings to store:
+  * Message ID mapping (Discourse chat message ID ↔ Discord message ID)
+  * User mapping (Discourse username ↔ Discord user/webhook representation)
+  * Channel mapping (Discourse channel ID ↔ Discord channel ID)
+  * Edit/delete operations require retrieving these mappings
+=== Event Filtering Strategy ===
+**Options:**
+  * "Send me everything" - receives all events (high bandwidth/processing)
+  * Selective events - choose specific event types
+  * Category/tag/group filters - sync only specific Discourse areas to specific Discord channels
+**Recommendation:** Use selective events and filtering to reduce noise and processing load
+=== Implementation Flow ===
+**Webhook Receiver (Discourse → Discord):**
+  - Receive webhook POST request
+  - Verify HMAC signature if secret configured
+  - Parse JSON payload
+  - Extract message/event data
+  - Check for existing mapping (to detect edits/deletes)
+  - Call Discord API
+  - Store message ID mapping
+**Discord Bot (Discord → Discourse):**
+  - Listen for Discord message events
+  - Check if message is from bot (skip to prevent loop)
+  - Format message for Discourse
+  - POST to Discourse API with credentials
+  - Store message ID mapping
+**Message History/Backfill:**
+  * Use ''GET /chat/api/channels/{channel_id}/messages'' for initial sync
+  * Implement pagination if needed
+  * Consider rate limiting
+===== Profile Image/Avatar Synchronization =====
+==== How Avatar Syncing Works ====
+The bridge synchronizes profile images/avatars bidirectionally between Discord and Discourse using webhook-based user impersonation.
+=== Discourse → Discord Avatar Flow ===
+  - Bridge polls Discourse Chat API every 1.5 seconds for new messages
+  - Each message includes user info with an ''avatar_template'' field
+  - Avatar template format: ''/user_avatar/site/username/{size}/123_2.png''
+  - Bridge calls ''build_avatar_url()'' to construct full URL:
+    * Replaces ''{size}'' placeholder with pixel size (default: 128)
+    * Prepends Discourse base URL if path is relative
+  - Full avatar URL passed to Discord webhook ''avatar_url'' parameter
+  - Discord displays the message with the Discourse user's actual avatar
+<code python>
+# Avatar URL construction (bridge.py lines 160-186)
+def build_avatar_url(self, avatar_template: str, size: int = 128) -> Optional[str]:
+    if not avatar_template:
+        return None
+    avatar_path = avatar_template.replace('{size}', str(size))
+    if not avatar_path.startswith('/'):
+        avatar_path = '/' + avatar_path
+    return f"{self.base_url}{avatar_path}"
+</code>
+=== Discord → Discourse Avatar Flow ===
+Discord user avatars are **not** synced to Discourse. Messages from Discord appear with the bot's configured username in Discourse, prefixed with ''[Discord] username:'' to identify the source user.
+=== User Impersonation ===
+The bridge uses Discord webhooks to display messages from Discourse users with their actual:
+  * **Username** - Discourse display name shown in Discord
+  * **Avatar** - Discourse profile image displayed in Discord
+This creates a seamless experience where Discourse users appear as distinct users in Discord channels.
+----
+===== Image Upload/Download Synchronization =====
+==== Discord → Discourse Images ====
+  - Discord attachments detected via ''message.attachments''
+  - Images downloaded to temporary files using ''download_image()''
+  - Uploaded to Discourse via ''/uploads.json?type=image'' endpoint
+  - Upload URL appended to message as markdown: ''![]({image_url})''
+  - Temporary files cleaned up after processing
+==== Discourse → Discord Images ====
+  - Images extracted from ''uploads'' field in Discourse messages
+  - Relative URLs converted to absolute URLs (prepends base_url)
+  - URLs appended to message content
+  - Discord auto-embeds images from URLs
+----
+===== GIF and Video Handling =====
+==== Giphy/Tenor URL Detection ====
+The bridge detects and converts Giphy/Tenor share URLs to direct media URLs for proper embedding:
+  * **Giphy**: ''https://giphy.com/gifs/...'' → ''https://media.giphy.com/media/{ID}/giphy.gif''
+  * **Tenor**: ''https://tenor.com/view/...'' → ''https://media.tenor.com/{ID}/tenor.gif''
+==== MP4 to GIF Conversion ====
+When Giphy/Tenor returns MP4 URLs (common for larger animations), the bridge converts them to GIF using FFmpeg:
+  - Downloads MP4 to temporary file
+  - Generates optimized color palette for compression
+  - Converts to GIF with configurable settings:
+    * Frame rate (default: 15 FPS)
+    * Width scaling (default: 480px)
+    * Duration limit (default: 10 seconds)
+    * CPU thread limit (default: 3 threads)
+  - Uploads converted GIF to Discourse
+  - Cleans up temporary files
+FFmpeg configuration options in ''bridge_config.json'':
+<code json>
+{
+  "bridge": {
+    "ffmpeg_threads": 3,
+    "ffmpeg_duration_limit": 10,
+    "ffmpeg_fps": 15,
+    "ffmpeg_scale_width": 480
+  }
+}
+</code>
+----
+===== State Management =====
+==== Message Tracking ====
+The bridge tracks processed message IDs to prevent duplicates and enable resumption after restarts.
+State file structure (''bridge_state.json''):
+<code json>
+{
+  "discord_channels": {
+    "channel_id": {
+      "last_message_id": 1234567890,
+      "last_updated": "2024-01-15T12:34:56Z"
+    }
+  },
+  "discourse_channels": {
+    "channel_id": {
+      "last_message_id": 456,
+      "last_updated": "2024-01-15T12:34:56Z"
+    }
+  }
+}
+</code>
+==== State Initialization ====
+On first run, the bridge initializes state with the **latest** message IDs to prevent flooding channels with historical messages.
+----
+===== Loop Prevention =====
+The bridge implements multiple safeguards to prevent infinite message loops:
+==== Discord → Discourse ====
+  * Messages prefixed with ''[Discord]'' tag
+  * Webhook messages (non-user) are detected and skipped
+  * Bot's own messages filtered by user ID
+==== Discourse → Discord ====
+  * Messages containing ''[Discord]'' prefix are ignored
+  * Prevents echoing messages that originated from Discord
+----
+===== Configuration Reference =====
+==== Full Configuration Structure ====
+<code json>
+{
+  "discord": {
+    "token": "bot_token_here",
+    "channels": [
+      {
+        "name": "channel_name",
+        "discord_channel_id": 123456789012345678,
+        "discourse_channel_id": 2,
+        "discord_webhook_url": null,
+        "discourse_webhook_url": "https://discourse.example.com/chat/hooks/UUID",
+        "enabled": true
+      }
+    ]
+  },
+  "discourse": {
+    "base_url": "https://discourse.example.com",
+    "api_key": "api_key_here",
+    "username": "syncbot"
+  },
+  "bridge": {
+    "polling_interval": 1.5,
+    "state_file": "bridge_state.json",
+    "log_level": "INFO",
+    "rate_limit_delay": 1.0,
+    "use_discord_webhooks": true,
+    "ffmpeg_threads": 3,
+    "ffmpeg_duration_limit": 10,
+    "ffmpeg_fps": 15,
+    "ffmpeg_scale_width": 480
+  }
+}
+</code>
+==== Configuration Options ====
+^ Option ^ Default ^ Description ^
+| ''polling_interval'' | 1.5 | Seconds between Discourse API polls |
+| ''state_file'' | bridge_state.json | Path to state persistence file |
+| ''log_level'' | INFO | Logging verbosity (DEBUG, INFO, WARNING, ERROR) |
+| ''rate_limit_delay'' | 1.0 | Minimum seconds between API calls |
+| ''use_discord_webhooks'' | true | Enable user impersonation via webhooks |
+| ''ffmpeg_threads'' | 3 | CPU threads for MP4 conversion |
+| ''ffmpeg_duration_limit'' | 10 | Max seconds of video to convert |
+| ''ffmpeg_fps'' | 15 | Frame rate for converted GIFs |
+| ''ffmpeg_scale_width'' | 480 | Width in pixels for converted GIFs |
+==== Environment Variable Overrides ====
+  * ''DISCORD_TOKEN'' - Discord bot token
+  * ''DISCOURSE_API_KEY'' - Discourse API key
+  * ''DISCOURSE_USERNAME'' - Discourse bot username
+  * ''DISCOURSE_BASE_URL'' - Discourse instance URL
+  * ''LOG_LEVEL'' - Logging level
+  * ''CONFIG_FILE'' - Path to config file
+  * ''STATE_FILE'' - Path to state file
+----
+===== Channel Mapping =====
+==== Multi-Channel Support ====
+The bridge supports multiple independent channel pairs. Each pair has:
+  * Discord channel ID
+  * Discourse channel ID
+  * Separate webhook URLs
+  * Independent enable/disable flag
+  * Separate state tracking
+==== Adding a New Channel Pair ====
+  - Add entry to ''discord.channels'' array in config
+  - Set ''discord_channel_id'' and ''discourse_channel_id''
+  - Create Discourse webhook and add URL to ''discourse_webhook_url''
+  - Set ''enabled: true''
+  - Restart bridge - Discord webhook created automatically
+----
+===== Error Handling and Retry Logic =====
+==== Automatic Retry ====
+  * Failed Discord webhook sends **do not** update state
+  * Message remains in queue and retries on next poll cycle
+  * Prevents message loss on temporary failures
+==== Graceful Degradation ====
+  * API errors caught and logged, processing continues
+  * Image download failures don't block message delivery
+  * FFmpeg conversion failures fall back to posting MP4 URL
+==== Timeouts ====
+  * API requests: 30-60 second timeout
+  * FFmpeg conversion: 30-60 second timeout
+  * Prevents indefinite hangs
+----
+===== Docker Deployment =====
+==== Container Configuration ====
+<code yaml>
+services:
+  syncbot:
+    build: .
+    restart: unless-stopped
+    deploy:
+      resources:
+        limits:
+          cpus: '1'
+          memory: 512M
+    logging:
+      driver: "json-file"
+      options:
+        max-size: "10m"
+        max-file: "3"
+    volumes:
+      - ./bridge-config.json:/app/bridge-config.json:ro
+      - ./bridge_state.json:/app/bridge_state.json
+</code>
+==== Security ====
+  * Runs as non-root user inside container
+  * Config mounted read-only
+  * State file writable for persistence
+  * Resource limits prevent runaway processes
+----
+===== Message Flow Diagrams =====
+==== Discord → Discourse ====
+<code>
+Discord User posts message
+         ↓
+on_message() event triggered
+         ↓
+Check: Is webhook message? → Skip
+Check: Is bot message? → Skip
+         ↓
+Find channel config by Discord channel ID
+         ↓
+Download image attachments (if any)
+         ↓
+Upload images to Discourse
+         ↓
+Extract/convert Giphy/Tenor URLs
+         ↓
+Convert MP4 to GIF (if needed)
+         ↓
+Format: "[Discord] username: message"
+         ↓
+Append image URLs as markdown
+         ↓
+Send via Discourse webhook
+         ↓
+Update Discord channel state
+</code>
+==== Discourse → Discord ====
+<code>
+poll_discourse() runs every 1.5 seconds
+         ↓
+For each enabled channel:
+         ↓
+GET messages after last_message_id
+         ↓
+For each new message:
+         ↓
+Check: Has [Discord] prefix? → Skip
+         ↓
+Extract username from user info
+Extract avatar_template from user info
+         ↓
+Build avatar URL (replace {size}, add base_url)
+         ↓
+Extract image uploads, make URLs absolute
+         ↓
+Truncate content to 2000 chars if needed
+         ↓
+Send via Discord webhook:
+  - username: Discourse username
+  - avatar_url: Discourse avatar
+  - content: message text
+         ↓
+Success? → Update Discourse channel state
+Failed? → Retry on next poll
+</code>
+----
+===== Dependencies =====
+==== Python Packages ====
+  * ''discord.py>=2.0.0'' - Discord bot framework
+  * ''requests>=2.28.0'' - HTTP requests
+  * ''aiohttp>=3.8.0'' - Async HTTP
+  * ''python-dotenv>=0.19.0'' - Environment variables
+==== System Dependencies ====
+  * Python 3.11+
+  * FFmpeg (for MP4 to GIF conversion)
+----
+===== Known Limitations =====
+==== Not Currently Implemented ====
+  * **Message Edit Sync** - Edits are not synchronized
+  * **Message Delete Sync** - Deletions are not synchronized
+  * **Reaction Sync** - Emoji reactions not bridged
+  * **PluralKit Support** - Removed in current version (was in earlier versions per git history)
+  * **Discord Avatar → Discourse** - Only Discourse avatars sync to Discord, not vice versa
+----
+===== Troubleshooting =====
+==== Messages Not Syncing ====
+  * Check ''enabled: true'' in channel config
+  * Verify Discord bot has webhook permissions in channel
+  * Check Discourse webhook URL is correct
+  * Review logs for API errors
+==== Duplicate Messages ====
+  * Check state file permissions (must be writable)
+  * Verify state file path matches config
+==== Images Not Appearing ====
+  * Verify Discourse API key has upload permissions
+  * Check temporary directory is writable
+  * Review logs for upload errors
+==== GIFs Not Converting ====
+  * Ensure FFmpeg is installed: ''ffmpeg -version''
+  * Check FFmpeg is in PATH
+  * Review logs for conversion errors
+  * Try reducing ''ffmpeg_duration_limit'' or ''ffmpeg_scale_width''