Building the Happy Thoughts platform: from Directus to a Django backend for 20+ apps

The setup: a CMS that was doing too much#

When I picked up Tejgyan Foundation’s Happy Thoughts platform in late 2022, the content backend was Directus — a headless CMS sitting on Postgres. For what it was doing at the time (courses, structured content, pages for the apps), Directus was the right call. Auto-generated CRUD, a usable admin, REST out of the box. You don’t hand-build that when a CMS hands it to you for free.

The strain showed up the moment the product needed the things a CMS is bad at: payment checkout, receipts, org accounts, progress and analytics. A CCAvenue integration, PDF receipts, transaction reconciliation — that’s application logic, not content modelling. You can bolt some of it onto a headless CMS with hooks and flows, but at that point you’re fighting the tool instead of using it.

The unconventional call: extend, don’t rewrite#

The conventional move here is “the CMS has hit its ceiling, let’s rewrite on Django.” I didn’t do that. A rewrite means a cutover, a freeze, and a stretch where the new thing is strictly worse than the old thing because it does less. On a non-profit’s live platform, with a small rotating team, that’s the fastest way to stall for six months and ship nothing.

So instead I deployed a Django/DRF service alongside Directus — pointed at the same Postgres database and the same Redis cluster. Same data, sibling subdomain. Directus kept serving content; Django quietly took the jobs Directus was bad at: payments, receipts, accounts, analytics. It landed on AWS ECS staging in December 2022 as, literally, an “extended-service.” Not a replacement — an extension.

That’s the strangler-fig pattern: wrap the old system, siphon capabilities into the new one a slice at a time, and let the old one keep running until it’s doing nothing that matters. Django absorbed the platform wave by wave until it owned the API and its own database. There’s a fossil that proves the history — the Redis cluster in today’s staging config is still named after Directus. Nobody renamed it, because nothing ever broke hard enough to force a cutover. That’s the whole point.

Retiring the admin without losing what it gave us#

Directus’s real value was never the content model — it was the free admin UI. Kill Directus and you lose auto-CRUD for every staff workflow overnight. So in April 2023 I stood up a react-admin panel on ra-data-django-rest-framework, which talks straight to DRF viewsets. That recreated Directus’s auto-CRUD value on top of our own API in weeks, not months — and every model we exposed through DRF got an admin screen almost for free. By 2025 the org had outgrown that too, and the admin became a Turborepo of per-product panels with a shared API client and X-Product header routing. But react-admin bought us two years of “add a viewset, get an admin” — a trade I’d make again.

Keycloak as the one front door#

By 2024 there were too many clients to keep authenticating ad hoc, so I moved the whole ecosystem onto Keycloak as org-wide IAM. The decision that raises eyebrows: no custom Django user model. AUTH_USER_MODEL points at KeycloakUser. Keycloak is the source of truth for identity; Django holds a synced shadow, kept fresh by a scheduled Celery task on its own queue. The Next.js apps relay tokens through NextAuth; django-uw-keycloak validates them on the backend. One realm, ~25 clients, one login.

Doing it without a custom user model isn’t the textbook Django answer. But a custom user model would’ve quietly made Django the identity authority — and it isn’t. Keycloak is. Modelling that honestly killed an entire category of who-owns-the-user sync bugs before they could exist.

Domain-sliced monolith, not microservices#

As surfaces multiplied — events, ERP, donations, room booking — the pull toward microservices gets strong. I kept the business logic in one Django monolith, sliced into apps: cms, accounts, iam, analytics, event, donations, erp, and so on. URL prefixes map 1:1 to apps (/cms/, /event/, /iam/). One deploy, one schema, transactions that actually span the domain when the business logic needs them to.

I only reached for a separate service where the runtime genuinely differs — GPU transcoding and ML embeddings — not because someone told me monoliths don’t scale.

Queue-per-workload Celery#

Async work runs on Celery over RabbitMQ, and here I was deliberate: a queue per workload. Receipt generation, progress tracking, audio transcode, video transcode, notifications, and user-preference creation each get a dedicated queue and worker. The reason is isolation — a slow batch of video transcodes must never sit in front of a donor’s receipt in the same queue. When we later built the access-management system, it got three queues of its own (provisioning, expiry, notifications) from day one, because I’d already been burned by shared queues elsewhere.

Media and AI without bloating the core#

2025 was the media-plus-AI year. yt-dlp/ffmpeg ingestion feeds the Celery transcode queues; HLS goes out via video.js. When transcoding got heavy I pulled it into a standalone FastAPI GPU service and did the cost engineering that a non-profit budget demands — RTX 4050-tuned ffmpeg images and a Vast.ai client that rents a spot GPU per job instead of keeping expensive instances warm.

The piece I’m proudest of is Hindi hybrid semantic search. Rather than stand up a separate vector database, I added pgvector columns to the Postgres we already run and put the embedding model in an external Indic-SBERT sidecar (l3cube-pune/indic-sentence-similarity-sbert, 768-dim). sentence-transformers got deliberately removed from Django’s dependencies — heavyweight ML has no business in the web container. Embeddings happen out of process, Postgres does the vector search, and a couple of SQL views exposed as unmanaged Django models give hybrid keyword-plus-vector search over transcribed Hindi content. Boring infrastructure, genuinely useful search.

What it adds up to#

Payments track the same graduate-when-you-must philosophy: CCAvenue in 2022 → Razorpay across checkout and donations → a RazorpayGateway adapter to formalize the abstraction → ICICI eNACH recurring mandates for regular donors. Each step earned its complexity by being needed, not anticipated.

Three and a half years in, one Django backend serves roughly 25 clients — ~20 Next.js apps, a react-admin panel, three native Android apps — for an LMS, events, donations, an ERP, room booking, and AI chat, all behind one Keycloak realm. The thing I’d underline for another engineer isn’t the Django backend itself. It’s that we never did a big-bang rewrite to get there. Knowing when to graduate off a tool, and doing it without a cutover, was the entire job.