Setu

You connect a source and a destination, pick the datasets to move, optionally transform them, and Setu runs the extract → transform → load — in the cloud, or on a runtime inside your own network when the data can’t leave it.

Setu (सेतु) means “bridge” — it bridges data between systems.

The problem

Moving data between systems is deceptively hard:

• Reach. The source or destination often lives where the cloud can’t see it — a private-VPC database, a folder on a laptop, an on-prem warehouse.
• Trust. Credentials for those systems shouldn’t have to be handed to a SaaS.
• Correctness. Runs must be resumable, cancellable, observable, idempotent — a migration that half-finishes silently is worse than one that fails loudly.

Setu answers these with a control-plane / data-plane split and a hybrid edge agent: the cloud orchestrates, while the actual data movement can run next to the data and report back over an outbound-only connection.

The big idea

• Control plane (apiserver + Postgres) owns connections, datasets, pipelines, runs, schedules, agents. It never touches edge credentials.
• Data plane (engine + worker + agent) does the actual E/T/L. The engine is a shared library; the worker runs it in the cloud; the agent runs the same engine near private data.
• Blob is the seam. Extract stages rows to an object store; load reads them back — so work can cross the cloud↔edge boundary without a direct connection.

Run an agent in one command

The agent opens no inbound ports. Enroll it against the control plane, then run it:

SETU_CONTROL_PLANE_URL=https://setu-api.onrender.com \
  setu enroll <token>
setu run
# → agent 09b8b0d7… polling https://setu-api.onrender.com

That’s the whole setup — no storage config, no VPN. The control plane hands the agent everything it needs (including where to stage) in each job. Read on for how the long-poll job loop actually works.