news-summary

# News Summary — AI 每日信息简报

**[English](#english) | [中文](#chinese)**

---

<a name="english"></a>
# 📰 News Summary — Daily AI Digest

An automated pipeline that aggregates your personal information sources every day, uses Claude AI to generate a structured digest, and delivers it to your inbox and Notion — all running on GitHub Actions for free.

## ✨ Features

| Feature | Description | Status |
|---------|-------------|--------|
| **30+ source types** | RSS, Substack, blogs, podcasts, YouTube transcripts, Twitter/X timeline, newsletters | ✅ |
| **Smart deduplication** | Tracks seen articles across days — no repeats | ✅ |
| **Age filtering** | Skips articles older than 30 days by default | ✅ |
| **Claude AI summaries** | Structured digest with highlights, bullet points, and source-by-source breakdown | ✅ |
| **Multilingual output** | Chinese (`zh`), English (`en`), or bilingual (`bilingual`) | ✅ |
| **Notion integration** | Creates a new page daily with clickable links; auto-archives duplicate pages if re-run on the same date | ✅ |
| **Email delivery** | Sends HTML email with clickable links to multiple recipients | ✅ |
| **GitHub Actions** | Runs automatically every day at 9:00 AM Beijing time — no server needed | ✅ |
| **No updates? Still shows** | Sources with no new content are listed as "no updates today" so you know they were checked | ✅ |
| **Telegram notifications** | Push digest to Telegram on completion | 🚧 untested |

## 🗂️ Supported Source Types

| Type | Examples |
|------|---------|
| `rss` | Substack, personal blogs, news sites, podcasts |
| `youtube_transcript` | YouTube channels — fetches full transcripts via Supadata API |
| `follow_builders_x` | Curated AI builders Twitter feed (no API key needed) |
| `follow_builders_podcasts` | Top AI podcasts with transcripts (no API key needed) |
| `twitter` | Your personal Twitter/X timeline via Playwright + cookies |
| `email` | Email newsletters via IMAP |
| `gmail` | Gmail via OAuth2 API |

## 🚀 Quick Start

### 1. Clone and install

```bash
git clone https://github.com/guo-yichen/news-summary.git
cd news-summary
pip install -r requirements.txt
```

### 2. Configure sources

```bash
cp sources.example.yaml sources.yaml
```

Edit `sources.yaml` with your sources. `sources.example.yaml` includes 30+ real curated sources (Anthropic, OpenAI, Google, Simon Willison, Latent Space, The Pragmatic Engineer, YC/VC blogs, and more) — all verified working. Use it as-is or as a starting point.

Set the output language at the top:
```yaml
language: zh        # zh | en | bilingual
```

### 3. Set environment variables

```bash
export ANTHROPIC_API_KEY=your-claude-api-key

# Notion (optional)
export NOTION_TOKEN=your-notion-token
export NOTION_DATABASE_ID=your-database-id

# Email (optional)
export EMAIL_USER=you@gmail.com
export EMAIL_PASSWORD=your-app-password   # Gmail app-specific password
export EMAIL_TO=you@gmail.com,other@example.com

# Telegram (optional)
export TELEGRAM_BOT_TOKEN=your-bot-token
export TELEGRAM_CHAT_ID=your-chat-id

# Output mode
export OUTPUT_MODE=both    # notion | markdown | both
```

### 4. Run

```bash
python -m src.runner
```

Depending on your configuration, the digest will be:
- Saved to `summaries/YYYY-MM-DD.md` (when `OUTPUT_MODE=markdown` or `both`)
- Written to Notion as a new page (when `OUTPUT_MODE=notion` or `both`)
- Sent to your inbox as an HTML email with clickable links (when `EMAIL_USER` + `EMAIL_PASSWORD` are set)

### 5. Twitter/X timeline (optional)

To include your personal Twitter timeline:

1. Install [Cookie-Editor](https://chrome.google.com/webstore/detail/cookie-editor/hlkenndednhfkekhgcdicdfddnkalmdm) browser extension
2. Log in to x.com
3. Click Cookie-Editor → Export → Export as JSON (copies to clipboard)
4. Run `pbpaste > twitter_cookies.json` in your terminal

Then add to `sources.yaml`:
```yaml
- type: twitter
  name: "My Twitter Timeline"
  max_tweets: 30
```

## ⚙️ GitHub Actions Setup

Fork this repo, then go to **Settings → Secrets and variables → Actions** and add:

| Secret | Description |
|--------|-------------|
| `SOURCES_YAML` | Base64-encoded content of your `sources.yaml` |
| `ANTHROPIC_API_KEY` | Claude API key |
| `NOTION_TOKEN` | Notion integration token |
| `NOTION_DATABASE_ID` | Notion database ID |
| `EMAIL_USER` | Gmail address for sending |
| `EMAIL_PASSWORD` | Gmail app-specific password |
| `EMAIL_TO` | Recipient email(s), comma-separated |
| `TELEGRAM_BOT_TOKEN` | Telegram bot token (optional) |
| `TELEGRAM_CHAT_ID` | Telegram chat ID (optional) |
| `SUPADATA_API_KEY` | Supadata API key for YouTube transcripts (optional) |
| `TWITTER_COOKIES` | Base64-encoded `twitter_cookies.json` (optional) |

Generate base64 for `sources.yaml`:
```bash
base64 -i sources.yaml | tr -d '\n' | gh secret set SOURCES_YAML
```

The workflow runs daily at **01:00 UTC (09:00 Beijing)**. You can also trigger it manually from the Actions tab.

## 🔧 Notion Setup

1. Go to [notion.so/my-integrations](https://www.notion.so/my-integrations) → New Integration → copy the token
2. Create a Notion database with a `title` property
3. Open the database → `...` → Connections → add your integration
4. Copy the database ID from the URL (the 32-character string after `notion.so/`)

## 📁 Project Structure

```
├── src/
│   ├── runner.py                  # Main entry: fetch → summarize → save
│   ├── summarize.py               # Claude AI summarization
│   ├── notion_writer.py           # Notion page writer
│   ├── email_sender.py            # Gmail SMTP sender
│   ├── telegram_notifier.py       # Telegram push
│   ├── state.py                   # Cross-day deduplication state
│   └── fetchers/
│       ├── base.py                # RawItem data structure
│       ├── rss.py                 # RSS fetcher
│       ├── follow_builders.py     # follow-builders Twitter + podcast feeds
│       ├── youtube_transcript.py  # YouTube transcript via Supadata
│       ├── twitter.py             # Twitter timeline via Playwright
│       ├── twitter_login.py       # Twitter login helper
│       └── email_fetcher.py       # IMAP / Gmail fetcher
├── .github/workflows/
│   └── daily-summary.yml          # GitHub Actions workflow
├── sources.example.yaml           # Source configuration template
└── requirements.txt
```

## 🙏 Credits

- [follow-builders](https://github.com/zarazhangrui/follow-builders) by [Zara Zhang](https://github.com/zarazhangrui) — the `follow_builders_x` and `follow_builders_podcasts` source types are inspired by and built on top of her centralized AI builders feed. No API key required thanks to her work.

---

<a name="chinese"></a>
# 📰 News Summary — AI 每日信息简报

每天自动聚合你关注的信息源，用 Claude AI 生成结构化简报，发送到你的邮箱和 Notion — 完全运行在 GitHub Actions 上，免费，无需服务器。

## ✨ 功能特性

| 功能 | 说明 | 状态 |
|------|------|------|
| **30+ 种信息源** | RSS、Substack、博客、播客、YouTube 字幕、Twitter/X 时间线、Newsletter | ✅ |
| **跨日去重** | 追踪已读文章，不重复推送 | ✅ |
| **时效过滤** | 默认跳过 30 天前的旧文章 | ✅ |
| **Claude AI 总结** | 结构化简报：今日要点 + bullet points + 来源详情 | ✅ |
| **多语言输出** | 中文（`zh`）、英文（`en`）、中英双语（`bilingual`）| ✅ |
| **Notion 集成** | 每天在 Notion 数据库创建新页面，链接可点击；同一天重复运行时自动归档旧页面，避免重复 | ✅ |
| **邮件推送** | 发送 HTML 格式邮件，链接可点击，支持多个收件人 | ✅ |
| **GitHub Actions** | 每天北京时间 9:00 自动运行，无需服务器 | ✅ |
| **无更新也显示** | 没有新内容的来源会标注「今日无新内容」，让你确认它被检查过 | ✅ |
| **Telegram 通知** | 生成完成后推送到 Telegram | 🚧 未测试 |

## 🗂️ 支持的信息源类型

| 类型 | 示例 |
|------|------|
| `rss` | Substack、博客、新闻网站、播客 |
| `youtube_transcript` | YouTube 频道（通过 Supadata API 获取完整字幕）|
| `follow_builders_x` | AI 大佬 Twitter 精选 Feed（无需 API key）|
| `follow_builders_podcasts` | 顶级 AI 播客字幕（无需 API key）|
| `twitter` | 你的个人 Twitter/X 时间线（Playwright + cookies）|
| `email` | 邮件 Newsletter（IMAP）|
| `gmail` | Gmail（OAuth2 API）|

## 🚀 快速开始

### 1. 克隆并安装依赖

```bash
git clone https://github.com/guo-yichen/news-summary.git
cd news-summary
pip install -r requirements.txt
```

### 2. 配置信息源

```bash
cp sources.example.yaml sources.yaml
```

编辑 `sources.yaml`，填入你要关注的信息源。`sources.example.yaml` 已内置 30+ 个真实可用的信

[truncated…]