From dfabe2e12f23fc6eb5ee8d3625c2e2d261f4e27b Mon Sep 17 00:00:00 2001 From: herbygitea Date: Wed, 25 Mar 2026 02:00:34 +0000 Subject: [PATCH] Create email-obsidian-git-pipeline.md via n8n --- .../Sessions/email-obsidian-git-pipeline.md | 146 ++++++++++++++++++ 1 file changed, 146 insertions(+) create mode 100644 PBS/Tech/Sessions/email-obsidian-git-pipeline.md diff --git a/PBS/Tech/Sessions/email-obsidian-git-pipeline.md b/PBS/Tech/Sessions/email-obsidian-git-pipeline.md new file mode 100644 index 0000000..813bbb9 --- /dev/null +++ b/PBS/Tech/Sessions/email-obsidian-git-pipeline.md @@ -0,0 +1,146 @@ +--- +project: email-obsidian-git-pipeline +type: session-notes +status: active +tags: + - pbs + - n8n + - automation + - gitea + - obsidian + - gmail + - gdrive +created: 2026-03-24 +updated: 2026-03-24 +path: PBS/Tech/Sessions/ +--- + +# Email to Obsidian Git Pipeline — Session Complete + +## 🎉 What We Built + +A fully automated document pipeline that takes Claude-drafted project emails and files them into Gitea (Obsidian vault) and Google Drive simultaneously. + +``` +Gmail → n8n → Parse Email → Extract Frontmatter + ↓ + Gitea (vault) ✅ + ↓ + GDrive (Jenny) ✅ + ↓ + Google Chat notification ✅ +``` + +--- + +## ✅ Completed This Session + +### Gmail OAuth & Trigger +- Created new OAuth 2.0 client in Google Console for n8n (separate from mail client OAuth) +- Enabled Gmail API and Google Drive API +- Gmail trigger polls every hour for unread emails matching subject filter +- Processed emails get labeled `n8n/processed` and removed from INBOX + +### Email Parsing +- HTML body extraction using `` tag to protect markdown formatting +- HTML entity cleaning and sanitization +- YAML frontmatter validation (checks for `---` prefix) + +### Frontmatter Extraction +- Custom `parseYamlValue()` function handles quoted and unquoted values +- Defensive defaults for missing fields: + - `project` defaults to slugified email subject (strips `[flags]`) + - `path` defaults to `PBS/Inbox` for malformed emails +- Subject flag extraction: `[flag1]` = trigger, `[flag2]` = routing category + +### Email Subject Convention +``` +[n8n] PBS Project - project name +[n8n] PBS Content - content idea +[n8n] Thought - quick capture +``` +- Gmail trigger filter: `subject:[n8n] -label:n8n/processed` +- Switch node routes based on `flag2` downstream + +### Gitea File Creation +- Gitea deployed on staging with SQLite (lightweight, no extra MySQL container) +- Healthcheck temporarily removed to allow initial setup via GUI +- Files written via Gitea REST API using HTTP Request node +- Auth: Header Auth with `token YOUR_TOKEN` format (not Bearer!) +- Internal Docker hostname: `http://gitea:3000/api/v1/` + +### Google Drive Upload +- Google Drive API enabled in Google Console +- Markdown converted to binary using Convert to File node (Move Base64 String to File operation) +- Files uploaded to `pbs-planning` folder +- Same markdown content used (raw text, not base64 like Gitea) + +### Google Chat Notifications +- Centralized notification sub-workflow with space selector +- Payload: `{ title, message, status, space, timestamp }` +- Message format: `🌻 PBS Message: *title*\nmessage\n\n_timestamp_` +- Error Trigger node catches failures, routes to same notification sub-workflow +- Spaces: `webadmin`, `insta`, `automation` + +--- + +## 🔑 Key Learnings + +- **Gitea auth** uses `token YOUR_TOKEN` not `Bearer` in Authorization header +- **GDrive Upload** requires binary data — use Convert to File node before GDrive node +- **Convert to File** operation must be `Move Base64 String to File` not `toText` +- **Google APIs** must be explicitly enabled per project even with OAuth credentials set up +- **n8n Error Trigger** only fires on production executions, not manual test runs +- **Gmail labels** are the underlying mechanism for folders — remove `INBOX` label to move emails +- **Gitea healthcheck** uses API endpoint that only responds after initial setup wizard completes — chicken and egg problem with Traefik! +- **Pinned data** in n8n doesn't always flow correctly through newly added nodes + +--- + +## 🔧 Gitea Setup Notes + +- Running on staging with SQLite database +- Admin user created via CLI: `docker exec -it -u git gitea gitea admin user create` +- Must use `-u git` flag or Gitea complains about running as root +- Initial healthcheck pointed to `/api/v1/version` which 404s before setup — temporarily removed to allow Traefik routing +- After setup complete, healthcheck can be restored + +--- + +## 📋 Next Steps + +- [ ] Restore Gitea healthcheck now that setup is complete +- [ ] Connect Obsidian Git plugin to Gitea repo on homelab +- [ ] Build Switch node routing for `flag2` (PBS Content, Thought, etc.) +- [ ] Migrate existing Google Chat webhook workflows to centralized sub-workflow +- [ ] Add GDrive dynamic subfolder creation based on `path` frontmatter field +- [ ] Deploy Gitea to production after staging validation +- [ ] Explore Quartz for browser-based vault viewing (Jenny access) +- [ ] Update Gmail trigger subject convention from `[PBS Project]` to `[n8n]` prefix + +--- + +## 🏗️ Current Pipeline Architecture + +``` +Gmail (hourly poll) + └── Filter: subject:[n8n] -label:n8n/processed + └── Parse HTML body (extract block) + └── Validate YAML frontmatter + └── Extract: project, path, flag2, emailId, subject + └── Prepare file content + ├── Gitea: base64 encoded → HTTP Request → /api/v1/repos/ + └── GDrive: base64 → Convert to File → GDrive Upload + └── Label email: add n8n/processed, remove INBOX + └── Notify: Google Chat automation space + +Error Trigger (unconnected, auto-fires on error) + └── Set error payload + └── Notify: Google Chat automation space +``` + +--- + +*Last Updated: 2026-03-24* +*Maintained by: Travis* +*Session: Email to Obsidian Git Pipeline* \ No newline at end of file