Create instagram-reel-sync-phase4.md via n8n

PBS/Tech/Projects/instagram-reel-sync-phase4.md

@@ -0,0 +1,223 @@
+---
+project: instagram-reel-sync-phase4
+type: project-plan
+status: active
+tags:
+  - pbs
+  - n8n
+  - instagram
+  - mysql
+  - automation
+created: 2026-03-17
+updated: 2026-03-17
+path: PBS/Tech/Projects/
+---
+
+# Phase 4 Handoff — Instagram Reel Detection & Sync
+
+## Context for New Chat Session
+
+This is a handoff document from the previous chat session. Paste this into the new
+chat along with the standard project context to get up to speed quickly.
+
+---
+
+## Current State — What's Already Built
+
+### MySQL Schema (pbs_automation database)
+
+**`pbs_recipes`:**
+```sql
+CREATE TABLE IF NOT EXISTS pbs_recipes (
+    id           BIGINT UNSIGNED NOT NULL AUTO_INCREMENT,
+    post_id      BIGINT UNSIGNED NOT NULL,
+    recipe_title        VARCHAR(500)    NOT NULL,
+    recipe_url          VARCHAR(500)    NOT NULL,
+    keyword      VARCHAR(100)    NULL,
+    created_at   DATETIME        DEFAULT CURRENT_TIMESTAMP,
+    updated_at   DATETIME        DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
+    PRIMARY KEY (id),
+    UNIQUE KEY (post_id)
+);
+```
+
+**`instagram_posts`:**
+```sql
+CREATE TABLE IF NOT EXISTS instagram_posts (
+    id           BIGINT UNSIGNED NOT NULL AUTO_INCREMENT,
+    reel_id      VARCHAR(50)     NOT NULL,
+    pbs_post_id      BIGINT UNSIGNED NULL,
+    instragram_caption  TEXT    NULL,
+    created_at   DATETIME        DEFAULT CURRENT_TIMESTAMP,
+    updated_at   DATETIME        DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
+    PRIMARY KEY (id),
+    UNIQUE KEY (reel_id),
+    FOREIGN KEY (post_id) REFERENCES pbs_recipes(post_id)
+);
+```
+
+**Key schema decisions:**
+- `post_id` is nullable in `instagram_posts` — NULL means unmatched reel
+- `reel_id` is VARCHAR not BIGINT (Instagram ID integer overflow issue)
+- Normalized — keyword and URL live in `pbs_recipes` only, JOIN at query time
+- Foreign key constraint means `pbs_recipes` record must exist before `instagram_posts` insert
+
+### WordPress → pbs_recipes Sync (Complete ✅)
+- WPCode Lite PHP snippet fires on every post save (admin only)
+- Payload: post_id, title, url, tags, categories
+- HMAC signature via X-PBS-Signature header (raw body via X-PBS-Body base64)
+- n8n workflow verifies signature, upserts into pbs_recipes
+- Keyword populated from first WordPress tag if present
+- COALESCE logic: only sets keyword if currently NULL (preserves manual edits)
+
+### Instagram Comment Reply Workflow (Complete ✅)
+- Handles incoming Instagram comment webhooks
+- Hash verification → subscribe check → PBS message filter
+- Looks up reel in `instagram_posts` table
+- Joins to `pbs_recipes` for keyword and URL
+- Sends DM + public reply on keyword match
+- All dead ends notify Travis via Google Chat
+
+### Notify Travis Workflow (Complete ✅)
+- Webhook trigger at: https://n8n.plantbasedsoutherner.com/webhook/notify-pbschat
+- Routes all alerts to Google Chat
+- Payload format:
+```json
+{
+  "title": "Alert title",
+  "message": "Detail message",
+  "timestamp": "={{ $now.toFormat('MMM dd, HH:mm:ss') }}"
+}
+```
+
+---
+
+## Phase 4 Goal — Automate Instagram Reel Detection
+
+### The Problem
+When Jenny publishes a new Instagram Reel, someone currently has to manually:
+1. Get the WordPress post ID
+2. Add it to the reel caption
+3. Add the reel ID to the `instagram_posts` table
+
+### The Solution — Three-Tier Detection System
+
+**Tier 1 — Comment Triggered (real time)**
+When a comment comes in and `instagram_posts` lookup returns empty:
+- Instead of just notifying Travis, trigger the "Fetch Reel Data" workflow
+- Pass enough context to reply to the original commenter if reel is found
+- First commenter may experience a few seconds delay — acceptable
+
+**Tier 2 — Scheduled Poll (every 12 hours)**
+- n8n Schedule Trigger polls Instagram media API
+- Catches reels that haven't received a comment yet
+- Proactively keeps table up to date
+
+**Tier 3 — Manual Trigger (Content Hub)**
+- "Sync Reels" button in PBS Content Hub Flask app
+- Jenny or assistant can trigger on demand after publishing
+
+### Queue and Retry Pattern (Option B)
+When Tier 1 triggers:
+1. Comment comes in → no record found → store original comment data
+2. Trigger "Fetch Reel Data" workflow with comment context
+3. If reel found → insert into `instagram_posts` → retry reply to original commenter
+4. If reel not found → Notify Travis with details
+
+---
+
+## Phase 4 Architecture
+
+### New Workflow — "Fetch Reel Data"
+
+**Triggers:**
+- Called from comment reply workflow (Tier 1) with single reel ID
+- Called from Schedule Trigger (Tier 2) with last 20 media items
+- Called from Content Hub webhook (Tier 3) manually
+
+**Core loop logic (Loop Over Items node):**
+```
+[Get reels from Instagram API]
+  → [Code Node: extract post_id from caption]
+  → [Loop Over Items]
+      → [MySQL SELECT: pbs_recipes WHERE post_id = extracted_id]
+      → [IF: record found?]
+          → TRUE → [MySQL UPSERT: instagram_posts]
+          → FALSE → [Notify Travis: Bad/missing post ID in caption]
+  → [After loop: if Tier 1 triggered → retry comment reply]
+```
+
+**Handoff payload from comment reply workflow (Tier 1):**
+```json
+{
+  "resolved_media_id": "the reel ID",
+  "comment_text": "original comment text",
+  "from_username": "commenter username",
+  "comment_id": "needed to post public reply"
+}
+```
+
+### Instagram Media API Details
+
+**No webhook exists for new reel publications** — confirmed via Meta docs research.
+Polling is the only option.
+
+**API endpoint:**
+```
+GET https://graph.facebook.com/v25.0/{ig-user-id}/media
+  ?fields=id,caption,media_type,media_product_type,permalink,timestamp
+  &limit=20
+  &access_token={token}
+```
+
+**Key field:** `media_product_type === "REELS"` — do NOT use `media_type`,
+it returns "VIDEO" for both regular videos and Reels.
+
+**Rate limits:** Generous — 4,800 × account impressions/24hrs. Polling every
+12 hours is negligible.
+
+**Token:** Long-lived user access token (60-day expiry). Build monitoring
+for token expiry — biggest operational risk.
+
+---
+
+## Key n8n Patterns Established
+
+- **Always Output Data** on MySQL nodes so IF nodes can evaluate empty results
+- **Check field exists** (`id exists`) not `length > 0` when Always Output Data is on
+- **Google Chat HTTP Request** must use "Using Fields Below" not raw JSON
+- **Message strings** built as single `={{ }}` expression with `+` concatenation
+- **`$now.toFormat()`** works in test and live contexts (not `$execution.startedAt`)
+- **MySQL query parameters** — do NOT use `=` prefix, plain comma separated values
+- **Loop Over Items** — use explicit `$('Node Name').first().json` not `$json`
+  inside loops since `$json` only references immediate previous node
+- **Set node before loops** — consolidate upstream data needed inside loop
+  so it's always accessible as `$json.field`
+- **`reel_id` always VARCHAR** — Instagram IDs exceed JS safe integer limit
+
+---
+
+## Remaining Work After Phase 4
+
+- [ ] Refactor comment reply workflow to split at `Check if we sent`
+      into separate "Instagram Reply Handler" workflow (cleaner logs)
+- [ ] PBS Content Hub — Phase 5 planning session required before building
+- [ ] Authelia SSO for admin tools (Portainer, n8n, Uptime Kuma, phpMyAdmin)
+- [ ] Deploy all staging changes to production
+
+---
+
+## Infrastructure Quick Reference
+
+- **Staging:** staging.plantbasedsoutherner.com
+- **n8n:** n8n.plantbasedsoutherner.com
+- **phpMyAdmin:** phpmyadmin.staging.plantbasedsoutherner.com
+- **PBS-API:** internal Docker service at http://pbs-api:5000
+- **MySQL database:** pbs_automation
+- **Google Chat alerts:** via Notify Travis workflow webhook
+
+---
+
+*Last Updated: March 17, 2026*
+*Maintained by: Travis*
+*Project: Plant Based Southerner — Instagram Automation Phase 4*