146 lines
5.2 KiB
Markdown
146 lines
5.2 KiB
Markdown
---
|
|
project: email-obsidian-git-pipeline
|
|
type: session-notes
|
|
status: active
|
|
tags:
|
|
- pbs
|
|
- n8n
|
|
- automation
|
|
- gitea
|
|
- obsidian
|
|
- gmail
|
|
- gdrive
|
|
created: 2026-03-24
|
|
updated: 2026-03-24
|
|
path: PBS/Tech/Sessions/
|
|
---
|
|
|
|
# Email to Obsidian Git Pipeline — Session Complete
|
|
|
|
## 🎉 What We Built
|
|
|
|
A fully automated document pipeline that takes Claude-drafted project emails and files them into Gitea (Obsidian vault) and Google Drive simultaneously.
|
|
|
|
```
|
|
Gmail → n8n → Parse Email → Extract Frontmatter
|
|
↓
|
|
Gitea (vault) ✅
|
|
↓
|
|
GDrive (Jenny) ✅
|
|
↓
|
|
Google Chat notification ✅
|
|
```
|
|
|
|
---
|
|
|
|
## ✅ Completed This Session
|
|
|
|
### Gmail OAuth & Trigger
|
|
- Created new OAuth 2.0 client in Google Console for n8n (separate from mail client OAuth)
|
|
- Enabled Gmail API and Google Drive API
|
|
- Gmail trigger polls every hour for unread emails matching subject filter
|
|
- Processed emails get labeled `n8n/processed` and removed from INBOX
|
|
|
|
### Email Parsing
|
|
- HTML body extraction using `` tag to protect markdown formatting
|
|
- HTML entity cleaning and sanitization
|
|
- YAML frontmatter validation (checks for `---` prefix)
|
|
|
|
### Frontmatter Extraction
|
|
- Custom `parseYamlValue()` function handles quoted and unquoted values
|
|
- Defensive defaults for missing fields:
|
|
- `project` defaults to slugified email subject (strips `[flags]`)
|
|
- `path` defaults to `PBS/Inbox` for malformed emails
|
|
- Subject flag extraction: `[flag1]` = trigger, `[flag2]` = routing category
|
|
|
|
### Email Subject Convention
|
|
```
|
|
[n8n] PBS Project - project name
|
|
[n8n] PBS Content - content idea
|
|
[n8n] Thought - quick capture
|
|
```
|
|
- Gmail trigger filter: `subject:[n8n] -label:n8n/processed`
|
|
- Switch node routes based on `flag2` downstream
|
|
|
|
### Gitea File Creation
|
|
- Gitea deployed on staging with SQLite (lightweight, no extra MySQL container)
|
|
- Healthcheck temporarily removed to allow initial setup via GUI
|
|
- Files written via Gitea REST API using HTTP Request node
|
|
- Auth: Header Auth with `token YOUR_TOKEN` format (not Bearer!)
|
|
- Internal Docker hostname: `http://gitea:3000/api/v1/`
|
|
|
|
### Google Drive Upload
|
|
- Google Drive API enabled in Google Console
|
|
- Markdown converted to binary using Convert to File node (Move Base64 String to File operation)
|
|
- Files uploaded to `pbs-planning` folder
|
|
- Same markdown content used (raw text, not base64 like Gitea)
|
|
|
|
### Google Chat Notifications
|
|
- Centralized notification sub-workflow with space selector
|
|
- Payload: `{ title, message, status, space, timestamp }`
|
|
- Message format: `🌻 PBS Message: *title*\nmessage\n\n_timestamp_`
|
|
- Error Trigger node catches failures, routes to same notification sub-workflow
|
|
- Spaces: `webadmin`, `insta`, `automation`
|
|
|
|
---
|
|
|
|
## 🔑 Key Learnings
|
|
|
|
- **Gitea auth** uses `token YOUR_TOKEN` not `Bearer` in Authorization header
|
|
- **GDrive Upload** requires binary data — use Convert to File node before GDrive node
|
|
- **Convert to File** operation must be `Move Base64 String to File` not `toText`
|
|
- **Google APIs** must be explicitly enabled per project even with OAuth credentials set up
|
|
- **n8n Error Trigger** only fires on production executions, not manual test runs
|
|
- **Gmail labels** are the underlying mechanism for folders — remove `INBOX` label to move emails
|
|
- **Gitea healthcheck** uses API endpoint that only responds after initial setup wizard completes — chicken and egg problem with Traefik!
|
|
- **Pinned data** in n8n doesn't always flow correctly through newly added nodes
|
|
|
|
---
|
|
|
|
## 🔧 Gitea Setup Notes
|
|
|
|
- Running on staging with SQLite database
|
|
- Admin user created via CLI: `docker exec -it -u git gitea gitea admin user create`
|
|
- Must use `-u git` flag or Gitea complains about running as root
|
|
- Initial healthcheck pointed to `/api/v1/version` which 404s before setup — temporarily removed to allow Traefik routing
|
|
- After setup complete, healthcheck can be restored
|
|
|
|
---
|
|
|
|
## 📋 Next Steps
|
|
|
|
- [ ] Restore Gitea healthcheck now that setup is complete
|
|
- [ ] Connect Obsidian Git plugin to Gitea repo on homelab
|
|
- [ ] Build Switch node routing for `flag2` (PBS Content, Thought, etc.)
|
|
- [ ] Migrate existing Google Chat webhook workflows to centralized sub-workflow
|
|
- [ ] Add GDrive dynamic subfolder creation based on `path` frontmatter field
|
|
- [ ] Deploy Gitea to production after staging validation
|
|
- [ ] Explore Quartz for browser-based vault viewing (Jenny access)
|
|
- [ ] Update Gmail trigger subject convention from `[PBS Project]` to `[n8n]` prefix
|
|
|
|
---
|
|
|
|
## 🏗️ Current Pipeline Architecture
|
|
|
|
```
|
|
Gmail (hourly poll)
|
|
└── Filter: subject:[n8n] -label:n8n/processed
|
|
└── Parse HTML body (extract block)
|
|
└── Validate YAML frontmatter
|
|
└── Extract: project, path, flag2, emailId, subject
|
|
└── Prepare file content
|
|
├── Gitea: base64 encoded → HTTP Request → /api/v1/repos/
|
|
└── GDrive: base64 → Convert to File → GDrive Upload
|
|
└── Label email: add n8n/processed, remove INBOX
|
|
└── Notify: Google Chat automation space
|
|
|
|
Error Trigger (unconnected, auto-fires on error)
|
|
└── Set error payload
|
|
└── Notify: Google Chat automation space
|
|
```
|
|
|
|
---
|
|
|
|
*Last Updated: 2026-03-24*
|
|
*Maintained by: Travis*
|
|
*Session: Email to Obsidian Git Pipeline* |