setu:: docs
The hybrid agent

Enrollment

Before an agent can pull work it needs a permanent key — but you never type one in. Enrollment is a one-time exchange: a short-lived token from the console is traded, exactly once, for a bearer key the agent keeps. This page traces that handshake end to end.

An agent authenticates every call with Authorization: Bearer <agentKey>. The problem enrollment solves is bootstrapping: how does a fresh binary on some host get that key without a human copying a secret around insecurely? The answer is a single-use, expiring token that is only useful for the one moment it's exchanged.

The agent's three states

An agent row starts life in enrolling when you register it in the console. It flips to active the instant a token is exchanged for a key, and can later be soft-revoked to revoked (its key stops authenticating). Only active agents can authenticate or be dispatched work.

apiserver/internal/db/migration/0023_agents.sql — agentsql
CREATE TABLE agent (
    id            uuid PRIMARY KEY DEFAULT gen_random_uuid(),
    workspace_id  uuid NOT NULL REFERENCES workspace(id) ON DELETE CASCADE,
    name          text NOT NULL,
    status        text NOT NULL DEFAULT 'enrolling'
                  CHECK (status IN ('enrolling','active','revoked')),
    key_hash      text,                       -- set at enrollment; sha-256 hex
    version       text NOT NULL DEFAULT '',
    last_seen_at  timestamptz,
    created_at    timestamptz NOT NULL DEFAULT now(),
    revoked_at    timestamptz
);

Note there is no plaintext key column — only key_hash, nullable because it doesn't exist until enrollment. The status CHECKmakes an illegal value unwritable, and the auth lookup filters on status = 'active', so a revoked (or still-enrolling) agent can never authenticate even if its hash matches.

Tokens and keys: 32 random bytes, stored only as a hash

Both the enrollment token and the agent key come from the same primitive: auth.NewToken() — 32 cryptographically-random bytes, base64url-encoded. Neither is ever stored in the clear. auth.HashToken() takes the sha-256 and hex-encodes it (64 chars), and that is what lands in the database. A database leak reveals only hashes, which can't be replayed as a credential.

apiserver/auth/auth.go — NewToken + HashTokengo
const tokenBytes = 32

// NewToken returns a fresh opaque token as a base64url string. The raw token is
// handed to the client once; only its HashToken value is ever persisted.
func NewToken() (string, error) {
	b := make([]byte, tokenBytes)
	if _, err := rand.Read(b); err != nil {
		return "", fmt.Errorf("auth: new token: %w", err)
	}
	return base64.RawURLEncoding.EncodeToString(b), nil
}

// HashToken returns the hex-encoded SHA-256 of a raw token. Storing the hash
// (never the raw token) means a database leak can't be replayed as a login.
func HashToken(raw string) string {
	sum := sha256.Sum256([]byte(raw))
	return hex.EncodeToString(sum[:])
}

The exchange, step by step

  1. 1
    Register the agent. An editor creates an agent in the console (registerAgent, editor+). The row is inserted in enrolling with no key yet.
  2. 2
    Mint a one-time token. The console calls the create-token endpoint. The server first confirms the agent exists in the caller's workspace (404 otherwise), generates a token, persists only its sha-256 hash with a 15-minute expiry, and returns the plaintext token once, wrapped in an install command.
  3. 3
    Run enroll on the host. You run setu enroll <token> next to your data, pointing at the control plane via SETU_CONTROL_PLANE_URL.
  4. 4
    Consume the token atomically. The server claims the token iff it is unconsumed and unexpired, stamping consumed_at in the same statement — so a replay finds no row and is rejected with 401.
  5. 5
    Mint and return the key. The server generates a fresh agent key, stores only its hash (flipping the agent to active and recording the reported version), and returns { agentId, agentKey, baseUrl } — the plaintext key exactly once.
  6. 6
    Persist config. The agent writes ~/.config/setu/config.json (mode 0600) with the key and the canonical baseUrl to poll, then you start setu run.

The handshake as a sequence

The enroll endpoint

POST/agent/v1/enrollpublic — the only unauthenticated /agent/v1 route
requestjson
{ "token": "<one-time enrollment token>", "version": "0.4.1" }
200 response (plaintext key, exactly once)json
{
  "agentId": "b1a2...",
  "agentKey": "sk_agent_...",
  "baseUrl":  "https://app.setu.example"
}
401 — replayed, expired, or unknown tokentext
{"error":"unauthenticated"}

Pointing at the control plane

Enrollment is the one flow that needs to know where the control plane is before any config exists. It reads SETU_CONTROL_PLANE_URL, defaulting to http://localhost:8080 for local dev.

SETU_CONTROL_PLANE_URLThe base URL setu enroll dials. Default http://localhost:8080.
SETU_CONFIGOverride the config file path. Default $HOME/.config/setu/config.json (legacy: SETU_AGENT_CONFIG).
SETU_PUBLIC_URLServer-side: the canonical public URL returned as baseUrl so the agent polls the right host. Default http://localhost:8080.

The agent side: client.Enroll + runEnroll

Enroll is a package-level helper (not a method) because the agent has no key yet — it's the only unauthenticated call. It POSTs the token and version, and decodes the three fields the control plane returns.

agent/internal/client/client.go — Enrollgo
// Enroll exchanges a one-time enrollment token for a permanent agent key. It is
// unauthenticated (the agent has no key yet). Returns the new agent id, key, and
// the canonical control-plane URL the agent should poll.
func Enroll(ctx context.Context, baseURL, token, version string) (agentID, agentKey, canonicalURL string, err error) {
	body, _ := json.Marshal(map[string]string{"token": token, "version": version})
	url := strings.TrimRight(baseURL, "/") + "/agent/v1/enroll"
	req, err := http.NewRequestWithContext(ctx, http.MethodPost, url, bytes.NewReader(body))
	// ...
	req.Header.Set("Content-Type", "application/json")
	resp, err := (&http.Client{Timeout: 30 * time.Second}).Do(req)
	// ... non-200 → statusError (reads a bounded 512-byte prefix of the body)
	var out struct {
		AgentID  string `json:"agentId"`
		AgentKey string `json:"agentKey"`
		BaseURL  string `json:"baseUrl"`
	}
	json.NewDecoder(resp.Body).Decode(&out)
	return out.AgentID, out.AgentKey, out.BaseURL, nil
}

runEnroll wires it up: it dials EnrollBaseURL(), then prefers the canonical URL the control plane reports over the one it dialed — so an agent enrolled via localhost or a LAN IP still polls home correctly. Then it saves the config.

agent/cmd/setu/main.go — runEnrollgo
func runEnroll(args []string) error {
	token := args[0]
	baseURL := config.EnrollBaseURL()

	ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
	defer cancel()

	agentID, agentKey, canonicalURL, err := client.Enroll(ctx, baseURL, token, agent.Version)
	if err != nil {
		return fmt.Errorf("enroll: %w", err)
	}
	// Poll the canonical URL the control plane reports, not necessarily the one
	// we dialed — so `setu run` reaches home even when enrolled via localhost/LAN.
	pollURL := baseURL
	if u := strings.TrimRight(strings.TrimSpace(canonicalURL), "/"); u != "" {
		pollURL = u
	}
	return config.Save(config.Config{
		BaseURL:  pollURL,
		AgentID:  agentID,
		AgentKey: agentKey,
		Version:  agent.Version,
	})
}

The server side: minting the token and the key

Two handlers bracket the exchange. createEnrollmentToken (editor+, workspace-scoped) mints the short-lived token; enroll (public) redeems it. On the mint side, only the hash is persisted with the 15-minute TTL, and the plaintext is returned exactly once, embedded in an install command the UI assembles.

apiserver/internal/api/agents.go — createEnrollmentToken (tail)go
const enrollTokenTTL = 15 * time.Minute

// ... confirm the agent exists in this workspace (404 otherwise) before issuing.
token, _ := auth.NewToken()
expiresAt := time.Now().Add(enrollTokenTTL)
s.q.CreateEnrollmentToken(r.Context(), sqlc.CreateEnrollmentTokenParams{
	TokenHash: auth.HashToken(token), AgentID: id, WorkspaceID: ws, ExpiresAt: expiresAt,
})
writeJSON(w, http.StatusCreated, map[string]any{
	"token":           token,     // plaintext, returned once
	"expiresAt":       expiresAt,
	"installCommand":  fmt.Sprintf("setu enroll %s", token),
	"controlPlaneUrl": controlPlaneURL(), // lets the UI fill URL-bearing snippets (e.g. Docker)
})

enroll is the only unauthenticated route under /agent/v1. The security-critical move is that consuming the token and minting the key are gated by an atomic ConsumeEnrollmentToken: if that returns no row, nothing is revealed.

apiserver/internal/api/agents.go — enrollgo
func (s *Server) enroll(w http.ResponseWriter, r *http.Request) {
	var req struct {
		Token   string `json:"token"`
		Version string `json:"version"`
	}
	// ... decode, require non-empty token (else 400)
	row, err := s.q.ConsumeEnrollmentToken(r.Context(), auth.HashToken(req.Token))
	if err != nil {
		s.unauthenticated(w) // invalid, expired, or already-consumed → reveal nothing
		return
	}
	key, _ := auth.NewToken()
	keyHash := auth.HashToken(key)
	s.q.SetAgentKey(r.Context(), sqlc.SetAgentKeyParams{
		ID: row.AgentID, KeyHash: &keyHash, Version: req.Version, // → status='active'
	})
	// Return the canonical public URL so the agent polls the right host no matter
	// which URL it dialed to enroll.
	writeJSON(w, http.StatusOK, map[string]any{
		"agentId":  row.AgentID,
		"agentKey": key,       // plaintext, returned exactly once
		"baseUrl":  controlPlaneURL(),
	})
}

The atomicity lives in one SQL statement. The UPDATE claims the token only if it is both unconsumed and unexpired, stamping consumed_at as it does — so two concurrent enrolls (or a replay) can't both win. The losing racer matches zero rows, :one returns pgx.ErrNoRows, and the handler answers 401.

apiserver/internal/db/query/agent.sql — ConsumeEnrollmentToken + SetAgentKeysql
-- name: ConsumeEnrollmentToken :one
-- Atomic one-time-use gate: claims the token iff it is unconsumed and unexpired,
-- stamping consumed_at in the same statement so a concurrent/replayed enroll
-- finds no row (→ rejected).
UPDATE agent_enrollment_token
SET consumed_at = now()
WHERE token_hash = $1 AND consumed_at IS NULL AND expires_at > now()
RETURNING agent_id, workspace_id;

-- name: SetAgentKey :exec
-- Activate an agent at enrollment: store the sha-256 hash of its key + version.
UPDATE agent
SET key_hash = $2, version = $3, status = 'active'
WHERE id = $1;
The key is shown once
Only the sha-256 hash of the agent key is ever stored (via SetAgentKey). The plaintext is in the enroll response and nowhere else. If the config is lost, you re-enroll with a fresh token rather than recover the old key. To rotate or retire a key, an editor calls revokeAgent: it sets status = 'revoked' and stamps revoked_at, and because the auth lookup filters on active, the old key stops authenticating immediately.

After enrollment: bearer auth on every call

With a key on disk, every subsequent request carries Authorization: Bearer <agentKey>. The routes live behind agentAuthMiddleware — a separate trust boundary from the browser session middleware. The middleware hashes the presented key, looks up an active agent by that hash, injects the agent id + workspace id into the request context, and best-effort touches last_seen_at so the console knows the agent is online.

apiserver/internal/api/api.go — agentAuthMiddlewarego
func (s *Server) agentAuthMiddleware(next http.Handler) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		key := bearerToken(r)
		if key == "" {
			s.unauthenticated(w) // header check runs before any DB lookup
			return
		}
		hash := auth.HashToken(key)
		agent, err := s.q.GetAgentByKeyHash(r.Context(), &hash) // WHERE key_hash=$1 AND status='active'
		if err != nil {
			s.unauthenticated(w) // unknown or revoked key → 401
			return
		}
		// Liveness is advisory: never fail the request if the touch write errors.
		_ = s.q.TouchAgentSeen(r.Context(), sqlc.TouchAgentSeenParams{ID: agent.ID, Version: agent.Version})
		ctx := context.WithValue(r.Context(), ctxKeyAgentID, agent.ID)
		ctx = context.WithValue(ctx, ctxKeyWorkspaceID, agent.WorkspaceID)
		next.ServeHTTP(w, r.WithContext(ctx))
	})
}

bearerToken parses the header defensively: it requires a case-insensitive Bearer prefix, trims surrounding space, and returns "" for anything malformed (a Basic header, a bare token, an empty value) — which the middleware turns into a 401 without ever touching the database. The authed route table:

GET/agent/v1/jobs/next?wait=<sec>Bearer agentKey

Long-poll for the next leased job. See Long-polling.

GET/agent/v1/jobs/{id}/extract-specBearer agentKey

Fetch the secretless extract instructions for a run-scoped extract job.

GET/agent/v1/jobs/{id}/load-specBearer agentKey

Fetch the secretless load instructions (incl. the pipeline document) for a load job.

POST/agent/v1/jobs/{id}/heartbeatBearer agentKey

Renew a leased job's lease while working. Always 204. See Lifecycle.

POST/agent/v1/jobs/{id}/resultBearer agentKey

Post a terminal result { status, result?, error? }. Agent-scoped: a job not owned by the caller 404s.

From here it's all the long-poll loop and the job lifecycle.