Three self-trained AI models read your mood from text, voice, or a selfie. A personalized Deezer playlist is generated, ranked by quality, and blended with your recent moods - in under two seconds, all open-source and privacy-first.
An intelligent music recommendation system that meets you where your mood is - typed, spoken, or photographed.
Make music discovery feel like being understood. Detect mood from three modalities (text, voice, face), then surface tracks that match the moment - not the demographic.
People express mood differently across contexts. A short text rarely lies; a voice clip captures tone you cannot type; a photo catches the micro-expression you would not admit to. Moodify reads all three.
Memory-snapshot inference container restores in seconds, not minutes. From input to playlist averages under two seconds end-to-end, even on first request after idle.
MIT license, public repo, public model weights on Hugging Face Hub. Every architectural decision is documented in the READMEs and condensed into the wiki below.
One React 18 codebase for web (Vercel), one Expo SDK 51 codebase for mobile (EAS Build). The mobile app talks directly to the Modal inference service - the Django API stays out of the data path.
Photos and voice clips are sent to the inference container for a single classification call and never persisted. Only the resulting mood label and saved track metadata land in your profile.
Every 👍, 👎, and open-in-Deezer trains your playlist ranker. A Thompson Sampling contextual bandit re-orders the list in real time; a per-user calibration map even corrects mis-classified emotions. New & anonymous users see the rule-based pipeline untouched.
Code, decisions, and tradeoffs are all on GitHub. Browse the issues, copy a pattern, file a PR. Moodify is a working reference for shipping AI products, not a black box.
A complete emotion detection and music recommendation system, surfaced through a clean, accessible UI.
Fine-tuned BERT classifier reads tone, sentiment, and emotional vocabulary from typed input. Five labels: sadness, joy, love, anger, fear.
SVC classifier over librosa MFCC features. Eight labels: calm, happy, sad, angry, fearful, disgust, surprised, neutral. Sub-second inference, sub-second feature extract.
FER Keras model wrapped in MTCNN face detection. Seven labels (Ekman + neutral). Works on a single front-camera photo, no video stream required.
Keyless integration with Deezer's Search API. Returns track name, artist, album, 30s preview, cover art, popularity rank, and a deep link to the streamable web player.
Thompson Sampling contextual bandit over a Beta-Bernoulli posterior. Every 👍 / 👎 / open-in-Deezer tap updates a 22-dim posterior (emotion × decade × duration × popularity-quintile) that re-ranks the next recommendation list. Cold-start safe: anonymous + new users see the base order until they cross 20 logged events.
A "Was this right?" widget captures every disagreement with the BERT detector. After three same-direction corrections (e.g. joy → love), the inference path rewrites the predicted label for that user only - immediate, no retrain, anonymous callers untouched.
Recency-weighted EWMA + first-order Markov chain + adaptive blend ratio produce the candidate list the bandit re-ranks. Recurring moods seep into the current playlist at a clamped, recency-decayed rate.
JWT auth (HS256, 7d access + 14d refresh), Mongo-backed profile, append-only mood + listening history, saved recommendations with rich track objects.
Sign in with Face ID, Touch ID, Windows Hello, or a hardware security key - no password. WebAuthn / FIDO2 verified by py_webauthn. Users enroll multiple passkeys and manage them (add / rename / delete) on a dedicated Account → Passkeys page; a verified assertion mints the same JWT pair as password login.
Expo SDK 51 + React Native 0.74 with Hermes engine. AsyncStorage for tokens, EAS Build for both stores, native camera + microphone for the three modalities.
Two-tier sliding-window rate limit (general + media) keyed on JWT sub, two-layer caches (text + Deezer + hash-of-bytes), and a hard MAX_CONTAINERS=5 ceiling at the Modal layer.
Per-request observability into MongoDB time-series collections. Admin-only /api/metrics/ aggregator returns p50/p95/p99 latency, error rates, and status-code breakdown for any rolling window.
Deezer down? Curated 14-track fallback list. Speech model fails to load? Endpoint returns degraded:true with neutral instead of 500. A failing model never takes down the others.
Theme switch toggles a full set of CSS variables. Honors prefers-color-scheme on first paint; persisted in localStorage across sessions.
17 country markets (Global + 16) and 19 genre filters bias the Deezer search phrase before it leaves the inference container. Results stay culturally relevant.
From "how do I feel" to a playlist on your phone in under two seconds.
Type a few words about your day, record a few seconds of your voice, or snap a quick selfie. Pick whichever feels easiest in the moment.
Three small AI models work behind the scenes - one for text, one for voice, one for faces. They turn your input into a mood label like "calm" or "excited."
Up to 60 tracks tuned to that mood appear instantly - with 30-second previews and one-tap deep links to Deezer. Save the ones you love to your history.
A side project that ended up being the music app I actually use every day.
No more endless scrolling for the right vibe. Three AI models read how you feel and skip straight to music that fits.
Selfies and voice clips are used once to detect mood, then thrown away. Only the mood label is saved. No tracking, no resale.
Under two seconds from input to a ranked, deduped 60-track playlist. No spinners, no waiting, no manual searching.
Every line of code is on GitHub. Read it, fork it, self-host it, contribute. Pull requests welcome.
Cloud-native, modular, and split along a clean service boundary. Pick a lens below.
auth.authenticate verifies either the JWT (end-user) or MODAL_SERVICE_TOKEN (Django proxy).mood_history and decides whether to fetch a second Deezer search for the recurring mood.rank_by_quality blends Deezer popularity with curated position; interleave mixes the two moods at the adaptive blend ratio.mood_history and recommendations on the Mongo profile. Then it routes to the Results page.DEPLOYMENT.md.k8s-addons/ directory layers in External Secrets, OPA Gatekeeper, Velero, and Chaos Mesh. See INFRASTRUCTURE_SETUP.md.JWT_SIGNING_KEY signs on Django and verifies on Modal. End-user tokens are 7d access + 14d refresh; silent refresh is wired into the React client.MODAL_SERVICE_TOKEN is a shared constant for the Django→Modal proxy path; bypasses the Modal rate limiter (DRF throttles that side).CsrfViewMiddleware is intentionally removed.MAX_CONTAINERS=5 Modal ceiling guarantee a worst-case capacity ceiling.Modern, production-ready, and selected for long-term maintainability over novelty.
Multiple paths. Pick the one that matches your risk and infra budget.
Two identical production environments behind one Service. Promote by flipping the selector label - instant traffic switch with zero downtime, instant rollback by flipping back.
Argo Rollouts gradually shifts traffic 10% → 25% → 50% → 100% with automated health checks at each step. Rollback is automatic on a failed gate.
GitHub Actions on the canonical path, Jenkins on the self-host path. Lint, test, security scan, image build, push, and Argo trigger. Quality gates + approvals on protected branches.
HPA, PDB, NetworkPolicy, and an Ingress with cert-manager. Auto-scaling and self-healing. The same Helm charts deploy to Kind locally, then EKS / GKE / OKE in production.
The recommended path. Git push → preview URL on every PR. modal deploy atomically swaps the inference container. Rollback by re-deploying a previous SHA.
Cluster watches the repo. Changes under argocd/applications/ reconcile automatically. No kubectl apply from laptops, no drift between Git and cluster.
Terraform modules under terraform/ spin up VPC + cluster + databases + monitoring for AWS / GCP / Azure / Oracle. One terraform apply away from a fresh environment.
Drop-in operators for the self-host path. External Secrets pulls from Vault, OPA Gatekeeper enforces policy, Velero handles backups, Chaos Mesh proves the system can take a punch.
Per-request rows to MongoDB TS (canonical) or kube-prometheus-stack with custom dashboards (self-host).
Vercel + Modal logs natively, plus Loki + Promtail with label-based querying on the K8s path.
Grafana Tempo on the K8s path; OpenTelemetry hooks ready in the Django middleware.
SLO PrometheusRule ships in the monitoring chart: latency, error rate, container budget. Slack / PagerDuty receivers wired in values.yaml.
A snapshot of what Moodify ships on the wire today.
Numbers measured against the production deploy.
Experience Moodify in action - or read the live API contract.
The full SPA with all three emotion modes, mood history, and the personalized Deezer playlist.
Launch App →Live Django REST API with OpenAPI 3 contract. Interactive Swagger UI for testing, Redoc for reading.
Swagger UI → Redoc →Open-source on GitHub with full READMEs per subsystem, OpenAPI YAML, Terraform, Helm, and CI/CD.
View on GitHub →FastAPI on Modal. Scale-to-zero, CPU memory snapshots, sliding-window rate limiting. GET /health for liveness.
Quick answers to what people ask most.
Most music apps ask you to pick a genre or playlist. Moodify reads your mood three different ways (text, voice, face) and tunes the music to fit - no scrolling, no decision fatigue, no manual searching.
Yes - an account keeps your mood history and saved tracks with you across visits and devices. Sign-up takes about 10 seconds: username, email, password. That's it.
Yes - Moodify supports passkeys. Right after sign-up (or anytime from Account → Passkeys) you can add a passkey and then sign in with Face ID, your fingerprint, Windows Hello, or a hardware security key - no password to type. You can add several (phone, laptop, security key), rename or remove them, and your password keeps working as a fallback.
Three ways - pick whichever you feel like. Type a few words about how you feel. Record a few seconds of your voice. Snap a quick selfie. Moodify reads any of those and finds music that matches.
No. Your photo or voice clip is used once to figure out the mood, then thrown away. Only the mood label itself ("happy", "calm", etc.) is kept so your history works. Nothing about your face or voice is stored.
Deezer. Every recommendation links straight to a streamable track on Deezer with a 30-second preview built in. Tap any track to open it on Deezer and listen.
Yes. You can bias the recommendations toward 19 different genres (lo-fi, EDM, k-pop, classical, hip-hop, etc.) and 17 different country markets if you want the catalog to feel local.
Moodify's inference container hibernates when nobody is using it. The first request after a quiet stretch takes about 1-2 seconds to wake the server up. Every request after that is fast.
Around 84-92% across the three modes on our test data. It works best on clear input: a normal-length sentence, a few seconds of audible voice, or a well-lit selfie facing the camera.
Yes - Moodify runs in any modern mobile browser, and there's a native iOS + Android app built with Expo. The mobile app uses the phone's camera and microphone directly.
No. The recommendation and mood-detection happen on a server, so an internet connection is required. The app itself opens fast, but you need to be online to get a playlist.
Yes - any time. The profile page has a Delete Account button. Your account, mood history, and saved recommendations are removed immediately.
It happens occasionally. Try a different mode (text vs voice vs face), or use the genre filter to push the playlist in the direction you actually want. The more you use Moodify, the better your personal blend gets.
Son Nguyen (@hoangsonww). Solo project, MIT-licensed, code is fully open on GitHub.
Open an issue on GitHub or email hoangson091104@gmail.com. Pull requests welcome.
Open the app, share how you feel, get a playlist tuned to the moment. Three modes, one ranked Deezer playlist, ready in seconds.