Contents
The deploy that wouldn’t deploy
I spent about fifteen months embedded on a venture-backed AI voice platform for home-services contractors — the kind of product that answers a plumber’s phone at 2am and books the job before the caller hangs up. Most of it ran serverless: Next.js on a serverless host, a couple thousand API routes, durable background jobs for the campaigns and CRM syncs. That model had carried the product for years, and I had zero interest in relitigating it.
Then a feature landed on my plate that needed to push a booking feed to a large search platform’s “book directly from search” program. The integration itself was unglamorous: generate a feed file, drop it on the partner’s SFTP endpoint on a schedule. SFTP. In 2026. Fine.
I reached for ssh2-sftp-client, the obvious library, wired it into a background job, and it did not run.
Why a serverless function can’t hold an SFTP session
There were two problems, and both are worth naming because they generalize well past this one feature.
First, an impedance mismatch in the execution model. A serverless function is request-scoped and ephemeral: it wakes up, does a thing, returns, and the runtime is then free to freeze or kill it. SFTP is the exact opposite — a stateful, connection-oriented protocol. You open a TCP socket, run the SSH handshake, negotiate a channel, stream the bytes, and tear it all down. That’s a long-lived conversation you want to babysit from a stable process, not something you want the platform freezing mid-handshake because your HTTP response already flushed.
Second, the dependency shape. ssh2-sftp-client sits on top of ssh2, which reaches for Node’s raw net sockets and crypto and pulls in optional native acceleration. The serverless bundler traces your imports to decide what to ship, and that tracing doesn’t reliably carry the pieces ssh2 wants at runtime. So even when the code reads correctly, the function comes up missing something at cold start.
You can burn a day fighting either of these. I know, because I started to.
Three doors
Once it’s clear the happy path won’t work, there are basically three ways out.
- Door 1 — fight the platform. Shim the missing deps, force a specific runtime, polyfill sockets, hand-pin the bundler’s file tracing. Maybe you win. But now you’ve got a fragile incantation that one dependency bump quietly un-does, and the next engineer has no idea why any of it is there.
- Door 2 — abandon serverless. “This proves serverless was a mistake, let’s containerize the whole thing.” One SFTP upload is a spectacularly expensive reason to move a codebase off the model it’s already shipping fine on.
- Door 3 — the escape hatch. Carve out one tiny service that runs on a real Node process, give it exactly this one job, and leave literally everything else serverless.
I took Door 3. The interesting part isn’t that choice, though — it’s where you draw the boundary once you’ve made it.
Draw the smallest possible boundary
This is the load-bearing idea, and it’s the part that’s easy to get wrong.
The moment you’re allowed to spin up “a service,” there’s a gravitational pull to let it grow. Give it a database connection. Let it figure out which tenants need a feed today. Let it own the schedule, the retries, the feature flags. Do that, and congratulations — you now have two platforms and a much worse problem than the one you started with.
The discipline is the opposite: the sidecar owns a capability, not a domain. Its entire job is “take these bytes, put them on that SFTP server.” It knows nothing about tenants, nothing about the data model, nothing about the product. All of that — which contractor, what’s in the feed, when it’s due — stays in the serverless app, which already has the tenancy, the auth, the database, and the durable-jobs system to do it well.
Concretely: the feed is generated where the data lives, and the sidecar only does transport.
// In the serverless app — it owns tenancy, data, auth. It does NOT do SFTP.
// GET /internal/feeds/booking -> returns the current feed file
export async function GET(req: Request) {
await verifyServiceAuth(req); // shared secret between app and sidecar
const feed = await buildBookingFeed(); // reads the DB, respects team scoping
return new Response(feed, {
headers: { "content-type": "application/xml" },
});
}
And the sidecar stays boring on purpose:
// apps/sftp-uploader — a pure-Node Express service. One job.
import express from "express";
import Client from "ssh2-sftp-client";
const app = express();
async function uploadFeed(): Promise<void> {
const res = await fetch(`${APP_URL}/internal/feeds/booking`, {
headers: { authorization: `Bearer ${SERVICE_TOKEN}` },
});
if (!res.ok) throw new Error(`feed fetch failed: ${res.status}`);
const body = Buffer.from(await res.arrayBuffer());
const sftp = new Client();
try {
await sftp.connect({
host: SFTP_HOST,
username: SFTP_USER,
privateKey: SFTP_PRIVATE_KEY,
});
await sftp.put(body, REMOTE_PATH); // stateful session, in a real process
} finally {
await sftp.end(); // always tear the socket down
}
}
app.get("/healthz", (_req, res) => res.json({ ok: true }));
app.listen(PORT);
Notice the shape. The sidecar has no DB client, no ORM, no tenant model. If someone tries to sneak “just a little” business logic in here, it sticks out immediately, because there’s nothing business-y to hide behind. That’s the whole point: a boundary you can see is a boundary you can defend in code review.
Where the cron lives — a small, wrong-looking call
Here’s the little decision that goes against the obvious move.
The platform had a perfectly good durable-jobs system for scheduled work. The tidy architecture-diagram answer is to trigger the upload from there and keep all scheduling in one place. I put the schedule in the sidecar instead — a plain daily timer inside the Node process.
Why? The entire reason this service exists is that it’s the one thing the durable-jobs system can’t run. Reaching back into that system to trigger the one job it can’t do re-couples the two things I just spent effort decoupling. A daily feed push is a single, self-contained transport job. Making it self-scheduling means the escape hatch is genuinely self-contained — you can reason about it, deploy it, and page on it in isolation, without so much as glancing at the main app’s job graph. One weird thing, fully encapsulated, is far easier to live with than one weird thing with a tendril reaching back into everything else.
On a system where the schedule needed to coordinate with other jobs, I’d make the opposite call and centralize it. This one didn’t, so I optimized for isolation on purpose.
What the escape hatch bought
The carve-out was small and it shipped fast — which was the whole appeal. No platform migration, no fragile bundler incantation, no relitigating a serverless decision that was working perfectly well. The main app stayed exactly as serverless as it had been the day before. What actually changed was that one capability — hold a stateful SFTP session on a schedule — moved to the one place that can do it, wrapped in the smallest surface I could get away with.
The instinct I’d hand to a peer: when your platform can’t do one specific thing, resist both reflexes — the urge to force the platform and the urge to throw the platform out. Find the single capability that doesn’t fit, give it the smallest possible home, and be ruthless about keeping every scrap of business logic on the other side of that line. An escape hatch is only a good idea for as long as it stays an escape hatch.