Spot the Scam

🏗️ Ensemble Architecture

• Calibrated Linear SVMs and Logistic Regression models
• TF-IDF vectorization for text features
• Engineered tabular features for job posting metadata
• Weighted ensemble combining top-performing models
• Isotonic calibration for reliable probability estimates

🤖 Machine Learning Models

• Classical Models: Logistic Regression, Linear SVM, XGBoost, LightGBM
• Deep Learning: DistilBERT fine-tuning for transformer-based classification
• Bayesian Hyperparameter Optimization via Optuna with TPE sampler
• Automated tuning workflows with YAML configuration overrides
• Model versioning and experiment tracking with MLflow

📊 Evaluation & Explainability

• Comprehensive metrics: F1, Precision, Recall, ROC-AUC, PR-AUC, Brier Score
• Calibration curves and Expected Calibration Error (ECE)
• Token-level contribution analysis with SHAP-style explanations
• Slice-based bias detection across job categories
• Automated report generation with visualizations

🚀 Deployment & API

• FastAPI REST endpoints for single and batch predictions
• Server-Sent Events (SSE) for streaming chat responses
• AI chatbot with Gemini integration for natural language queries
• Docker containerization for reproducible environments
• ONNX model export for cross-platform compatibility

📈 Model Calibration

Our model demonstrates excellent calibration with an Expected Calibration Error (ECE) of only 0.0066 on the test set, ensuring that predicted probabilities accurately reflect true confidence levels.

• Decision Threshold: 0.5802 (optimized on validation F1)
• Gray-Zone Width: 0.10 (for uncertain predictions)
• Calibration Method: Isotonic Regression
• Expected Calibration Error (Test): 0.0066

📂 Repository Structure

spot-the-scam/
├── 📦 src/spot_scam/       # Core Python package
│   ├── data/                # Data ingestion & preprocessing
│   ├── features/            # Feature engineering (TF-IDF, tabular)
│   ├── models/              # Classical, XGBoost, Transformer models
│   ├── tuning/              # Optuna hyperparameter optimization
│   ├── evaluation/          # Metrics, curves, calibration, reporting
│   ├── inference/           # FraudPredictor, gray-zone policy
│   └── api/                 # FastAPI endpoints & schemas
├── 🎨 frontend/             # Next.js dashboard + chat UI
├── 🔧 configs/              # YAML configuration files
├── 📜 scripts/              # CLI utilities (train, tune, API)
├── 💾 artifacts/            # Trained models, vectorizers, metadata
├── 📊 experiments/          # Generated reports, figures, tables
├── 🧪 tests/                # Unit tests
├── 🐳 docker/               # Dockerfiles & compose configs
└── 📖 docs/                 # Comprehensive documentation

🐍 Backend & ML

⚛️ Frontend

🚀 DevOps & Infrastructure

🤖 AI & LLM Integration

🐳 Docker Quick Start

# Build and run with Docker Compose
docker-compose up --build

# Access the services
# API: http://localhost:8000
# Frontend: http://localhost:3000
# MLflow: http://localhost:5000

⚠️ Important: The hosted demo at Vercel runs with fake data and a demo model for exploration. For a fully functional version with real predictions, please follow the Docker setup instructions to run locally.

BibTeX

@software{spot_the_scam_2025-2026,
  title = {Spot the Scam: Calibrated Job-Posting Fraud Detection},
  author = {Nguyen, Son},
  year = {2025-2026},
  version = {0.1.0},
  url = {https://github.com/hoangsonww/Spot-the-Scam-AI-Job-Fraud},
  note = {End-to-end ML pipeline for detecting fraudulent job postings with 
          classical and transformer models, calibrated policies, and 
          explainability tooling.}
}

APA

Nguyen, S. (2025-2026). Spot the Scam: Calibrated Job-Posting Fraud Detection (Version 0.1.0) [Computer software]. https://github.com/hoangsonww/Spot-the-Scam-AI-Job-Fraud

MIT License

This project is licensed under the MIT License. You are free to use, modify, and distribute this software for both commercial and non-commercial purposes. See the LICENSE file for details.