Create email-obsidian-git-pipeline.md via n8n
This commit is contained in:
parent
856e5a8291
commit
dfabe2e12f
146
PBS/Tech/Sessions/email-obsidian-git-pipeline.md
Normal file
146
PBS/Tech/Sessions/email-obsidian-git-pipeline.md
Normal file
@ -0,0 +1,146 @@
|
||||
---
|
||||
project: email-obsidian-git-pipeline
|
||||
type: session-notes
|
||||
status: active
|
||||
tags:
|
||||
- pbs
|
||||
- n8n
|
||||
- automation
|
||||
- gitea
|
||||
- obsidian
|
||||
- gmail
|
||||
- gdrive
|
||||
created: 2026-03-24
|
||||
updated: 2026-03-24
|
||||
path: PBS/Tech/Sessions/
|
||||
---
|
||||
|
||||
# Email to Obsidian Git Pipeline — Session Complete
|
||||
|
||||
## 🎉 What We Built
|
||||
|
||||
A fully automated document pipeline that takes Claude-drafted project emails and files them into Gitea (Obsidian vault) and Google Drive simultaneously.
|
||||
|
||||
```
|
||||
Gmail → n8n → Parse Email → Extract Frontmatter
|
||||
↓
|
||||
Gitea (vault) ✅
|
||||
↓
|
||||
GDrive (Jenny) ✅
|
||||
↓
|
||||
Google Chat notification ✅
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ✅ Completed This Session
|
||||
|
||||
### Gmail OAuth & Trigger
|
||||
- Created new OAuth 2.0 client in Google Console for n8n (separate from mail client OAuth)
|
||||
- Enabled Gmail API and Google Drive API
|
||||
- Gmail trigger polls every hour for unread emails matching subject filter
|
||||
- Processed emails get labeled `n8n/processed` and removed from INBOX
|
||||
|
||||
### Email Parsing
|
||||
- HTML body extraction using `` tag to protect markdown formatting
|
||||
- HTML entity cleaning and sanitization
|
||||
- YAML frontmatter validation (checks for `---` prefix)
|
||||
|
||||
### Frontmatter Extraction
|
||||
- Custom `parseYamlValue()` function handles quoted and unquoted values
|
||||
- Defensive defaults for missing fields:
|
||||
- `project` defaults to slugified email subject (strips `[flags]`)
|
||||
- `path` defaults to `PBS/Inbox` for malformed emails
|
||||
- Subject flag extraction: `[flag1]` = trigger, `[flag2]` = routing category
|
||||
|
||||
### Email Subject Convention
|
||||
```
|
||||
[n8n] PBS Project - project name
|
||||
[n8n] PBS Content - content idea
|
||||
[n8n] Thought - quick capture
|
||||
```
|
||||
- Gmail trigger filter: `subject:[n8n] -label:n8n/processed`
|
||||
- Switch node routes based on `flag2` downstream
|
||||
|
||||
### Gitea File Creation
|
||||
- Gitea deployed on staging with SQLite (lightweight, no extra MySQL container)
|
||||
- Healthcheck temporarily removed to allow initial setup via GUI
|
||||
- Files written via Gitea REST API using HTTP Request node
|
||||
- Auth: Header Auth with `token YOUR_TOKEN` format (not Bearer!)
|
||||
- Internal Docker hostname: `http://gitea:3000/api/v1/`
|
||||
|
||||
### Google Drive Upload
|
||||
- Google Drive API enabled in Google Console
|
||||
- Markdown converted to binary using Convert to File node (Move Base64 String to File operation)
|
||||
- Files uploaded to `pbs-planning` folder
|
||||
- Same markdown content used (raw text, not base64 like Gitea)
|
||||
|
||||
### Google Chat Notifications
|
||||
- Centralized notification sub-workflow with space selector
|
||||
- Payload: `{ title, message, status, space, timestamp }`
|
||||
- Message format: `🌻 PBS Message: *title*\nmessage\n\n_timestamp_`
|
||||
- Error Trigger node catches failures, routes to same notification sub-workflow
|
||||
- Spaces: `webadmin`, `insta`, `automation`
|
||||
|
||||
---
|
||||
|
||||
## 🔑 Key Learnings
|
||||
|
||||
- **Gitea auth** uses `token YOUR_TOKEN` not `Bearer` in Authorization header
|
||||
- **GDrive Upload** requires binary data — use Convert to File node before GDrive node
|
||||
- **Convert to File** operation must be `Move Base64 String to File` not `toText`
|
||||
- **Google APIs** must be explicitly enabled per project even with OAuth credentials set up
|
||||
- **n8n Error Trigger** only fires on production executions, not manual test runs
|
||||
- **Gmail labels** are the underlying mechanism for folders — remove `INBOX` label to move emails
|
||||
- **Gitea healthcheck** uses API endpoint that only responds after initial setup wizard completes — chicken and egg problem with Traefik!
|
||||
- **Pinned data** in n8n doesn't always flow correctly through newly added nodes
|
||||
|
||||
---
|
||||
|
||||
## 🔧 Gitea Setup Notes
|
||||
|
||||
- Running on staging with SQLite database
|
||||
- Admin user created via CLI: `docker exec -it -u git gitea gitea admin user create`
|
||||
- Must use `-u git` flag or Gitea complains about running as root
|
||||
- Initial healthcheck pointed to `/api/v1/version` which 404s before setup — temporarily removed to allow Traefik routing
|
||||
- After setup complete, healthcheck can be restored
|
||||
|
||||
---
|
||||
|
||||
## 📋 Next Steps
|
||||
|
||||
- [ ] Restore Gitea healthcheck now that setup is complete
|
||||
- [ ] Connect Obsidian Git plugin to Gitea repo on homelab
|
||||
- [ ] Build Switch node routing for `flag2` (PBS Content, Thought, etc.)
|
||||
- [ ] Migrate existing Google Chat webhook workflows to centralized sub-workflow
|
||||
- [ ] Add GDrive dynamic subfolder creation based on `path` frontmatter field
|
||||
- [ ] Deploy Gitea to production after staging validation
|
||||
- [ ] Explore Quartz for browser-based vault viewing (Jenny access)
|
||||
- [ ] Update Gmail trigger subject convention from `[PBS Project]` to `[n8n]` prefix
|
||||
|
||||
---
|
||||
|
||||
## 🏗️ Current Pipeline Architecture
|
||||
|
||||
```
|
||||
Gmail (hourly poll)
|
||||
└── Filter: subject:[n8n] -label:n8n/processed
|
||||
└── Parse HTML body (extract block)
|
||||
└── Validate YAML frontmatter
|
||||
└── Extract: project, path, flag2, emailId, subject
|
||||
└── Prepare file content
|
||||
├── Gitea: base64 encoded → HTTP Request → /api/v1/repos/
|
||||
└── GDrive: base64 → Convert to File → GDrive Upload
|
||||
└── Label email: add n8n/processed, remove INBOX
|
||||
└── Notify: Google Chat automation space
|
||||
|
||||
Error Trigger (unconnected, auto-fires on error)
|
||||
└── Set error payload
|
||||
└── Notify: Google Chat automation space
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
*Last Updated: 2026-03-24*
|
||||
*Maintained by: Travis*
|
||||
*Session: Email to Obsidian Git Pipeline*
|
||||
Loading…
Reference in New Issue
Block a user