Create email-to-obsidian-automation.md via n8n
This commit is contained in:
parent
041172df32
commit
1ebd2b8bd9
208
PBS/Tech/Projects/email-to-obsidian-automation.md
Normal file
208
PBS/Tech/Projects/email-to-obsidian-automation.md
Normal file
@ -0,0 +1,208 @@
|
|||||||
|
---
|
||||||
|
project: email-to-obsidian-automation
|
||||||
|
type: project-plan
|
||||||
|
status: active
|
||||||
|
tags:
|
||||||
|
- pbs
|
||||||
|
- n8n
|
||||||
|
- automation
|
||||||
|
- docker
|
||||||
|
- traefik
|
||||||
|
- production
|
||||||
|
created: 2026-03-19
|
||||||
|
updated: 2026-03-19
|
||||||
|
path: PBS/Tech/Projects/
|
||||||
|
---
|
||||||
|
|
||||||
|
# Email-to-Obsidian Automation — Project Plan
|
||||||
|
|
||||||
|
## 🎯 Project Goal
|
||||||
|
|
||||||
|
Automate the pipeline from Claude chat session → Obsidian vault. Currently,
|
||||||
|
Claude drafts project notes as `[PBS Project]` emails to
|
||||||
|
tjcherb@plantbasedsoutherner.com via Gmail draft or message_compose. Those
|
||||||
|
emails are sitting unprocessed in Gmail. This project builds an n8n
|
||||||
|
workflow that watches for those emails, parses the Obsidian-compatible
|
||||||
|
markdown (including YAML frontmatter), commits the `.md` file to a Gitea
|
||||||
|
repo, and lets the Obsidian Git plugin sync it down to the homelab vault.
|
||||||
|
|
||||||
|
**Full pipeline:**
|
||||||
|
Claude drafts email → Gmail → n8n picks it up → parses frontmatter → writes
|
||||||
|
.md to Gitea → Obsidian Git plugin pulls to homelab vault
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📋 Prerequisites
|
||||||
|
|
||||||
|
- [ ] Linode PBS server running (Docker, Traefik, n8n — already operational)
|
||||||
|
- [ ] Google Workspace email (tjcherb@plantbasedsoutherner.com — already
|
||||||
|
operational)
|
||||||
|
- [ ] Obsidian vault on homelab (already exists)
|
||||||
|
- [ ] Cloudflare DNS with wildcard (already configured)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🗓️ Phase 1: Gitea Setup on Linode
|
||||||
|
|
||||||
|
### Goal
|
||||||
|
Deploy Gitea as a lightweight, self-hosted Git server on the PBS Linode
|
||||||
|
server. Accessible via Traefik at `git.plantbasedsoutherner.com` (or
|
||||||
|
similar subdomain). This becomes the central Git remote for the Obsidian
|
||||||
|
vault and potentially other PBS projects (Ansible playbooks, PBS-API code,
|
||||||
|
Content Hub, n8n workflow exports, dotfiles).
|
||||||
|
|
||||||
|
### Step 1.1: Add Gitea to Docker Compose
|
||||||
|
- [ ] Add Gitea service to compose stack (use `gitea/gitea:latest` image)
|
||||||
|
- [ ] Provision a MySQL database (or SQLite if keeping it lean — decision
|
||||||
|
point)
|
||||||
|
- [ ] Mount persistent volume for Gitea data (`/data`)
|
||||||
|
- [ ] Connect to the Traefik network
|
||||||
|
- [ ] Add Traefik labels for routing (`git.plantbasedsoutherner.com`)
|
||||||
|
- [ ] Deploy and verify Gitea web UI is accessible with valid SSL
|
||||||
|
|
||||||
|
### Step 1.2: Configure Gitea
|
||||||
|
- [ ] Complete initial setup wizard (admin account, settings)
|
||||||
|
- [ ] Create a `pbs-obsidian-vault` repository
|
||||||
|
- [ ] Generate an API token or SSH key for n8n to push commits
|
||||||
|
- [ ] Store credentials in ansible-vault (consistent with existing secrets
|
||||||
|
management)
|
||||||
|
|
||||||
|
### Step 1.3: Initialize the Vault Repo
|
||||||
|
- [ ] Clone the empty repo to homelab
|
||||||
|
- [ ] Copy existing Obsidian vault contents into the repo
|
||||||
|
- [ ] Initial commit and push to Gitea
|
||||||
|
- [ ] Verify files visible in Gitea web UI
|
||||||
|
|
||||||
|
### Step 1.4: Obsidian Git Plugin Setup (Homelab)
|
||||||
|
- [ ] Install the Obsidian Git community plugin
|
||||||
|
- [ ] Configure remote pointing to Gitea repo
|
||||||
|
- [ ] Set auto-pull interval (every 10-15 minutes)
|
||||||
|
- [ ] Enable auto-push for local edits (two-way sync)
|
||||||
|
- [ ] Test: manually create a file in Gitea web UI → verify it appears in
|
||||||
|
Obsidian after pull
|
||||||
|
|
||||||
|
### Decision Points
|
||||||
|
- **Subdomain:** `git.plantbasedsoutherner.com` vs `
|
||||||
|
gitea.plantbasedsoutherner.com`?
|
||||||
|
- **Database:** SQLite (simpler, fine for light use) vs MySQL (consistent
|
||||||
|
with existing stack)?
|
||||||
|
- **Ansible:** Add Gitea to existing Ansible playbook now, or manual deploy
|
||||||
|
first then codify later?
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🗓️ Phase 2: n8n Email-to-Vault Workflow
|
||||||
|
|
||||||
|
### Goal
|
||||||
|
Build an n8n workflow that watches Gmail for `[PBS Project]` emails,
|
||||||
|
extracts the markdown content, parses the YAML frontmatter to determine the
|
||||||
|
file path and name, and commits the `.md` file to the correct location in
|
||||||
|
the Gitea vault repo.
|
||||||
|
|
||||||
|
### Step 2.1: Gmail Trigger Setup
|
||||||
|
- [ ] Add a Gmail trigger node in n8n (poll-based or webhook)
|
||||||
|
- [ ] Filter for emails with subject prefix `[PBS Project]`
|
||||||
|
- [ ] Extract the email body (HTML — the markdown is wrapped in ``
|
||||||
|
tags based on current format)
|
||||||
|
- [ ] Mark or label processed emails to avoid re-processing (e.g., add a
|
||||||
|
`processed` Gmail label)
|
||||||
|
|
||||||
|
### Step 2.2: Parse Email Content
|
||||||
|
- [ ] Strip HTML wrapper (`` tags) to get raw markdown
|
||||||
|
- [ ] Extract YAML frontmatter block (content between `---` delimiters)
|
||||||
|
- [ ] Parse frontmatter fields:
|
||||||
|
- `project` → used for filename (e.g.,
|
||||||
|
`instagram-automation-session-notes.md`)
|
||||||
|
- `path` → target directory in vault (e.g., `PBS/Tech/Sessions/`)
|
||||||
|
- `type` → informational, no routing logic needed yet
|
||||||
|
- `status` → informational
|
||||||
|
- `tags` → informational
|
||||||
|
- `created` / `updated` → informational
|
||||||
|
- [ ] Construct full file path: `{path}/{project}.md`
|
||||||
|
- [ ] Handle edge cases:
|
||||||
|
- Missing frontmatter → notify Travis via Google Chat, skip processing
|
||||||
|
- Duplicate project slug → overwrite (update) existing file (desired
|
||||||
|
behavior for status changes)
|
||||||
|
|
||||||
|
### Step 2.3: Commit to Gitea via API
|
||||||
|
- [ ] Use Gitea's REST API to create/update the file in the repo
|
||||||
|
- `POST /api/v1/repos/{owner}/{repo}/contents/{filepath}` for new files
|
||||||
|
- `PUT` with SHA for updates to existing files
|
||||||
|
- [ ] Set commit message: `[n8n] Add/Update: {project slug} ({type})`
|
||||||
|
- [ ] Authenticate using the API token from Phase 1
|
||||||
|
|
||||||
|
### Step 2.4: Notification & Error Handling
|
||||||
|
- [ ] On success: Google Chat notification — `✅ Vault updated: {project} →
|
||||||
|
{path}`
|
||||||
|
- [ ] On failure (parse error, API error, missing frontmatter): Google Chat
|
||||||
|
alert with details
|
||||||
|
- [ ] Wire all dead-end branches to notifications (same pattern as
|
||||||
|
Instagram workflow)
|
||||||
|
|
||||||
|
### Step 2.5: Backfill Existing Emails
|
||||||
|
- [ ] Run the workflow manually against existing `[PBS Project]` emails
|
||||||
|
sitting in Gmail
|
||||||
|
- [ ] Verify all files land in correct vault paths
|
||||||
|
- [ ] Spot-check frontmatter parsing on a few files
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🧪 Testing Strategy
|
||||||
|
|
||||||
|
- [ ] Test on staging n8n instance first (consistent with existing workflow)
|
||||||
|
- [ ] Use pinned execution data for safe testing
|
||||||
|
- [ ] Send a test email with known frontmatter → verify file appears in
|
||||||
|
Gitea
|
||||||
|
- [ ] Verify Obsidian Git plugin picks up the new file on homelab
|
||||||
|
- [ ] Test update scenario: send email with same project slug → verify file
|
||||||
|
is overwritten
|
||||||
|
- [ ] Test error scenario: send email with malformed/missing frontmatter →
|
||||||
|
verify Google Chat alert
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🔑 Key Architecture Decisions
|
||||||
|
|
||||||
|
- **Gitea on Linode (same server as n8n):** Enables localhost
|
||||||
|
communication, no external auth tokens needed for n8n → Gitea. Also useful
|
||||||
|
for other PBS repos beyond just the vault.
|
||||||
|
- **Gitea REST API (not Git CLI from n8n):** Cleaner than shelling out to
|
||||||
|
git commands inside the n8n container. API is well-documented and n8n HTTP
|
||||||
|
Request node handles it easily.
|
||||||
|
- **Obsidian Git plugin (not cron job):** Simpler setup, two-way sync, only
|
||||||
|
runs when Obsidian is open (which is when you need the files anyway). Can
|
||||||
|
add cron later if needed.
|
||||||
|
- **Overwrite on duplicate slug:** Treat the `project` field as a unique
|
||||||
|
key. If Claude sends an updated version of a project plan, the new one
|
||||||
|
replaces the old. Git history preserves all versions.
|
||||||
|
- **Gmail label for dedup:** Processed emails get labeled so the trigger
|
||||||
|
doesn't re-process them.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📊 Success Criteria
|
||||||
|
|
||||||
|
- [ ] Gitea running at chosen subdomain with valid SSL
|
||||||
|
- [ ] Obsidian vault repo syncing bidirectionally (Gitea ↔ homelab)
|
||||||
|
- [ ] n8n workflow triggers on new `[PBS Project]` emails
|
||||||
|
- [ ] Markdown files land in correct vault paths based on frontmatter
|
||||||
|
- [ ] Existing backlog of Gmail project emails are processed
|
||||||
|
- [ ] Error handling wired to Google Chat
|
||||||
|
- [ ] End-to-end test: Claude drafts email → file appears in Obsidian vault
|
||||||
|
within 15 minutes
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🎯 Future Enhancements (Not in Scope)
|
||||||
|
|
||||||
|
- [ ] Add Gitea to Ansible playbook for reproducible deploys
|
||||||
|
- [ ] Authelia SSO for Gitea web UI (bundle with Portainer/Uptime Kuma SSO
|
||||||
|
project)
|
||||||
|
- [ ] n8n workflow version control — export workflow JSON to Gitea repo
|
||||||
|
- [ ] PBS-API and Content Hub source code hosted on Gitea
|
||||||
|
- [ ] Obsidian vault backup strategy (Gitea → offsite mirror)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
*Last Updated: March 19, 2026*
|
||||||
|
*Maintained by: Travis*
|
||||||
Loading…
Reference in New Issue
Block a user