Prototype on Dify, Ship on Celery

There’s a piece of engineering folklore that says “don’t build throwaway infrastructure.” I broke that rule on purpose, and it turned out to be one of the better calls I made across a four-year project.

The feature was an AI deal analyser for a startup-fundraising platform: drop in a pitch deck, get back a structured ten-section investment report — market, team, traction, competitive landscape, the works. The hard part on day one wasn’t the code. It was figuring out what a good report even said. That’s a product problem wearing an engineering costume, and you don’t solve it by writing Celery tasks. You solve it by iterating on prompts in front of the people who’ll read the output, as fast as you can change them.

So I reached for a visual workflow tool — self-hosted Dify — knowing full well I’d throw most of it away. Two months later I did exactly that and rewrote the pipeline as native Django and Celery code. This post is about why both halves of that were the right move, and how to tell when you’ve crossed from one to the other.

What a visual workflow tool actually buys you#

Dify — or n8n, Flowise, LangFlow, pick your poison — gives you a canvas: nodes for LLM calls, branching, variable passing, and HTTP tool calls, wired together visually. The thing it really sells is iteration speed on the parts of a pipeline that are still in flux.

For the deal analyser that meant two things:

1. I could reshape the report — reorder sections, rewrite a prompt, add a “red flags” pass — and see the result in seconds, with no deploy. Stakeholders were in the room. That feedback loop is the whole point.
2. External data — company research, founder background, web results — plugged in as HTTP tools instead of getting baked into the flow.

That second point is worth dwelling on, because it’s the pattern that survives the migration. A visual tool can’t scrape a profile or hit a SERP proxy on its own, but it can call an HTTP endpoint. So I stood up a tiny FastAPI sidecar whose entire job was to wrap the messy scraping and research calls behind clean endpoints the workflow could treat as tools.

# tool_sidecar/main.py — a scraping step, exposed as an HTTP "tool"
from fastapi import FastAPI
from pydantic import BaseModel

app = FastAPI()

class ScrapeRequest(BaseModel):
    url: str

@app.post("/scrape")
def scrape(req: ScrapeRequest) -> dict:
    # rotating-key pool in front of a rate-limited, quota-bound upstream
    key = key_pool.next_available()
    html = fetch_via_unlocker(req.url, api_key=key)
    return {"text": extract_readable(html)}

The sidecar had exactly one reason to exist: keep the slow, flaky, quota-bound parts of the world behind a stable contract. That key-rotation detail matters — third-party scraping APIs will happily 429 you into the ground, so a pool that rolls to the next available key is the difference between a demo that works and one that dies mid-pitch. And that stable contract is what let me swap the orchestration layer later without touching the data-gathering layer.

The catch: everything a canvas hides#

A visual workflow is a great place to discover a pipeline and a terrible place to own one. The moment the deal analyser stopped being an experiment and became a thing customers relied on, every property I’d traded away started to hurt:

• Prompts lived inside the tool’s database, not in git. No diffs, no code review, no “why did this change?” Ten sections of carefully tuned instructions, and the only record of how they’d evolved was my memory.
• Testing meant clicking. There’s no pytest for a canvas. I couldn’t assert that a tweak to the “traction” prompt didn’t quietly break the “risks” section.
• It was another platform to run. Another container, another thing to back up, upgrade, and page someone about at 2 a.m.
• Multi-tenancy fought the model. The platform was whitelabel — each tenant picks their own LLM vendor and brings their own keys. Threading per-tenant vendor selection through a single shared visual workflow is possible and miserable.

None of these are Dify’s fault. They’re the cost of any visual tool once the thing it orchestrates becomes core. The tool did its job — it let me find the shape of the report. Keeping it past that point would have been mistaking the scaffolding for the building.

Graduating to Django and Celery#

The migration had three moving parts: get the prompts into version control, replace the visual DAG with Celery orchestration, and make the LLM vendor a per-tenant runtime choice.

1. A prompt library that’s just files#

The least clever decision here was the best one. Instead of a prompt-management SaaS or a database table, each report section became a flat .txt file in the repo:

deal_analyser/prompts/
├── 01_market.txt
├── 02_team.txt
├── 03_traction.txt
├── ...
└── 10_recommendation.txt

Loading them is boring on purpose:

from pathlib import Path
from functools import cache

PROMPT_DIR = Path(__file__).parent / "prompts"

@cache
def load_prompt(section: str) -> str:
    return (PROMPT_DIR / f"{section}.txt").read_text()

def render(section: str, **context: str) -> str:
    return load_prompt(section).format(**context)

Boring is the feature. Now a prompt change is a pull request. It diffs. Someone reviews it. If the “recommendation” section starts hallucinating funding rounds, git blame tells me which commit introduced it and which reviewer waved it through. That’s the single biggest thing version control bought back — prompts are the actual product logic of an LLM feature, and they deserve the same rigor as the code around them, not a text box in someone else’s admin panel.

2. Celery instead of a canvas#

The visual DAG was really just “gather the data, run ten independent section analyses, then assemble.” That maps onto Celery almost one-to-one — and the ten sections are embarrassingly parallel, which a chord expresses cleanly:

from celery import shared_task, chord

SECTIONS = ["01_market", "02_team", "03_traction", ...]  # 10 total

@shared_task(bind=True, max_retries=3, default_retry_delay=10)
def analyse_section(self, context: dict, section: str, tenant_id: int) -> dict:
    client = llm_client_for(tenant_id)
    prompt = render(section, **context)
    return {"section": section, "body": client.complete(prompt)}

@shared_task
def assemble_report(results: list[dict], deck_id: int) -> None:
    ordered = sorted(results, key=lambda r: r["section"])
    DealReport.objects.create(deck_id=deck_id, sections=ordered)

def run_deal_analysis(deck_id: int, tenant_id: int) -> None:
    context = gather_research(deck_id)   # calls the same sidecar as before
    header = [analyse_section.s(context, s, tenant_id) for s in SECTIONS]
    chord(header)(assemble_report.s(deck_id))

The things I got back for free by moving here: retries with backoff on a flaky section, rate limiting per tenant, visibility in the same monitoring I already had for every other background job, and the ten sections actually running in parallel instead of a canvas stepping through nodes one at a time.

3. Per-tenant LLM, resolved at runtime#

Because the sidecar had kept data gathering behind a stable contract, the only genuinely new orchestration code was vendor resolution — and the whitelabel product needed it anyway:

def llm_client_for(tenant_id: int) -> LLMClient:
    tenant = SiteCompany.objects.get(pk=tenant_id)
    # one interface over Claude (default), GPT, Gemini, Perplexity
    return LLMClient(vendor=tenant.llm_vendor, keys=tenant.keys)

In Dify this would have meant per-tenant workflow copies or a thicket of conditional nodes. In code it’s a function call. A tenant switches from Claude to GPT by changing a config field, and every one of the ten section tasks picks it up on the next run.

How to know when the visual tool is done#

The switch from Dify to Celery took about two months of calendar time, but knowing when to make it was the actual skill. My rule of thumb, earned on this project:

Stay on the visual tool while the answer to “what should this do?” is still changing faster than “how should this run?” Graduate the moment those flip.

Concretely, I moved when I could tick these off:

• The output’s shape had stopped changing — ten sections, stable order. I was tuning wording, not structure.
• Someone outside the experiment depended on it. That’s when testability and version control stop being nice-to-haves.
• I needed a property the canvas couldn’t give me — here, per-tenant vendor selection and diffable prompts.
• I was spending more time operating the tool than iterating inside it.

If you’re still discovering the pipeline, migrating early just means you’ll rebuild it three more times in code. If you’ve stopped discovering and you’re still on the canvas, you’re paying rent on flexibility you no longer use.

The part that isn’t about tools#

The reason both halves worked is that I treated the visual tool as disposable from day one. I never let production data model itself around Dify, never let it hold state nothing else could see, and kept the expensive, external parts — the scraping and research — behind an HTTP contract that outlived the orchestrator. That discipline is what turned the graduation into a two-month project instead of a from-scratch rewrite.

“Don’t build throwaway infrastructure” is decent advice right up until the throwaway is the cheapest way to answer a question you can’t answer any other way. Build it, get your answer, and have the discipline to tear it down when the question changes from what to how. Prototype on the visual tool. Ship on the boring one.