First off, thank you for considering contributing to Skill Seeker! It's people like you that make Skill Seeker such a great tool.
⚠️ IMPORTANT: Skill Seekers uses a two-branch workflow.
main (production)
↑
│ (only maintainer merges)
│
development (integration) ← default branch for PRs
↑
│ (all contributor PRs go here)
│
feature branches
main - Production branch
development by maintainersdevelopment - Integration branch
main by maintainersFeature branches - Your work
developmentadd-github-scraping)development via PR# 1. Fork and clone
git clone https://github.com/YOUR_USERNAME/Skill_Seekers.git
cd Skill_Seekers
# 2. Add upstream
git remote add upstream https://github.com/yusufkaraaslan/Skill_Seekers.git
# 3. Create feature branch from development
git checkout development
git pull upstream development
git checkout -b my-feature
# 4. Make changes, commit, push
git add .
git commit -m "Add my feature"
git push origin my-feature
# 5. Create PR targeting 'development' branch
This project and everyone participating in it is governed by our commitment to fostering an open and welcoming environment. Please be respectful and constructive in all interactions.
Before creating bug reports, please check the existing issues to avoid duplicates.
When creating a bug report, include:
Example:
**Bug:** MCP tool fails when config has no categories
**Steps to Reproduce:**
1. Create config with empty categories: `"categories": {}`
2. Run `python3 cli/doc_scraper.py --config configs/test.json`
3. See error
**Expected:** Should use auto-inferred categories
**Actual:** Crashes with KeyError
**Environment:**
- OS: Ubuntu 22.04
- Python: 3.10.5
- Version: 1.0.0
Enhancement suggestions are tracked as GitHub issues.
Include:
We welcome new framework configurations! To add one:
configs/Example PR:
**Add Svelte Documentation Config**
Adds configuration for Svelte documentation (https://svelte.dev/docs).
- Config: `configs/svelte.json`
- Tested with max_pages: 100
- Successfully categorized: getting_started, components, api, advanced
- Total pages available: ~150
We actively welcome your pull requests!
⚠️ IMPORTANT: All PRs must target the development branch, not main.
developmentdevelopment branch!Fork and clone the repository
git clone https://github.com/YOUR_USERNAME/Skill_Seekers.git
cd Skill_Seekers
Install dependencies
pip install requests beautifulsoup4
pip install pytest pytest-cov
pip install -r mcp/requirements.txt
Create a feature branch from development
git checkout development
git pull upstream development
git checkout -b feature/my-awesome-feature
Make your changes
# Edit files...
Run tests
python -m pytest tests/ -v
Commit your changes
git add .
git commit -m "Add awesome feature"
Push to your fork
git push origin feature/my-awesome-feature
Create a Pull Request
python -m pytest tests/ -v)## Description
Brief description of what this PR does.
## Type of Change
- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
- [ ] Documentation update
## How Has This Been Tested?
Describe the tests you ran to verify your changes.
## Checklist
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my feature works
- [ ] New and existing unit tests pass locally with my changes
We follow PEP 8 with some modifications:
snake_casePascalCaseUPPER_SNAKE_CASE# 1. Standard library imports
import os
import sys
from pathlib import Path
# 2. Third-party imports
import requests
from bs4 import BeautifulSoup
# 3. Local application imports
from cli.utils import open_folder
# 4. Constants
MAX_PAGES = 1000
DEFAULT_RATE_LIMIT = 0.5
# 5. Functions and classes
def my_function():
"""Docstring describing what this function does."""
pass
Add comments for complex logic
def scrape_page(url: str, selectors: dict) -> dict:
"""
Scrape a single page and extract content.
Args:
url: The URL to scrape
selectors: Dictionary of CSS selectors
Returns:
Dictionary containing extracted content
Raises:
RequestException: If page cannot be fetched
"""
pass
# Run all tests
python -m pytest tests/ -v
# Run specific test file
python -m pytest tests/test_mcp_server.py -v
# Run with coverage
python -m pytest tests/ --cov=cli --cov=mcp --cov-report=term
tests/ directorytest_Use descriptive test names
def test_config_validation_with_missing_fields():
"""Test that config validation fails when required fields are missing."""
config = {"name": "test"} # Missing base_url
result = validate_config(config)
assert result is False
Skill_Seekers/
├── cli/ # CLI tools
│ ├── doc_scraper.py # Main scraper
│ ├── package_skill.py # Packager
│ ├── upload_skill.py # Uploader
│ └── utils.py # Shared utilities
├── mcp/ # MCP server
│ ├── server.py # MCP implementation
│ └── requirements.txt # MCP dependencies
├── configs/ # Framework configs
├── docs/ # Documentation
├── tests/ # Test suite
└── .github/ # GitHub config
└── workflows/ # CI/CD workflows
Releases are managed by maintainers:
Contributors will be recognized in:
Thank you for contributing to Skill Seeker! 🎉