RSS Crawler

GitHub
MCP-RSS-Crawler

An MCP (Message Chain Protocol) server that fetches RSS feeds and shares them with LLMs.

Features

  • Fetching and caching of RSS feeds (SQLite database)
  • MCP protocol implementation for seamless LLM integration
  • Support for filtering feeds by category, source, or keywords
  • Comprehensive API endpoints for feed management
    • Add, update, and delete feeds
  • Support for fetching articles from Firecrawl

Requirements

  • Bun
  • Firecrawl API key
  • Claude Desktop or other MCP client

Setup as MCP Server

  1. Clone this repository
  2. Create a claude_desktop_config.json file based on claude_desktop_config.json.example with your configuration
{
  "mcpServers": {
    "rss-crawler": {
      "command": "/path/to/bun",
      "args": ["run", "/path/to/mcp-rss-crawler/apps/mcp-server/src/mcp-cli.ts"],
      "cwd": "/path/to/mcp-rss-crawler",
      "env": {
        "PORT": "5556",
        "DB_DIR": "/path/to/mcp-rss-crawler",
        "FIRECRAWL_API_KEY": "fc-<YOUR_FIRECRAWL_API_KEY>"
      }
    }
  }
}
  1. Install dependencies: 
    bun install
  2. Start Claude Desktop:

MCP Protocol

The server implements the Message Chain Protocol (MCP) which allows LLMs to access your latest RSS feeds. The MCP endpoint accepts POST requests with a JSON body containing a messages array and returns a response with the latest feed items.

Example request:

{
  "messages": [
    {
      "role": "user",
      "content": "What are the latest news from my RSS feeds?"
    }
  ]
}

Example response:

{
  "messages": [
    {
      "role": "assistant",
      "content": "Here are the latest articles from your RSS feeds:",
      "name": "rss-mcp"
    },
    {
      "role": "tool",
      "content": "[{\"title\":\"Article Title\",\"summary\":\"Article summary...\",\"published\":\"2025-03-16T04:30:00.000Z\",\"origin\":\"Feed Name\",\"link\":\"https://example.com/article\"}]",
      "name": "rss-feeds"
    }
  ]
}

Configuration Options

The server can be configured through environment variables or a .env file:

  • PORT - Server port (default: 5556)
  • FIRECRAWL_API_KEY - Firecrawl API key
  • DB_DIR - Database directory (default: ~/.mcp-rss-crawler)

Troubleshooting

  • For connection issues, check your network settings and firewall configuration
  • Logs are available in the console and can be used to diagnose problems
  • For more detailed logging, set the DEBUG=mcp-rss:* environment variable