Redis Worker Queue

This guide builds a job processing system made of two independent services: job-api accepts HTTP requests and pushes job IDs onto a Redis list; job-worker pops from that list and processes jobs in the background. The two services are deployed, scaled, and restarted independently. They share a Redis connection string, but each service holds its own copy of the secret.

Overview

The architecture is straightforward:

• job-api — HTTP service. Accepts POST /enqueue, pushes a job ID to a Redis list, returns 202.
• job-worker — Background service. BLPOPs from the same Redis list and processes each job.

Both services run on Satusky as separate deployments. You can scale the worker without touching the API, take the worker down for maintenance while the API keeps accepting jobs (Redis holds them), and ship changes to either service independently.

Secrets are scoped per deployment. REDIS_URL must be added to each service separately — they don’t share a secret store. When you destroy a deployment, its secrets are destroyed with it. If you redeploy from scratch, you must re-add the secrets.

Project structure

task-system/
├── api/
│   ├── app.py
│   ├── requirements.txt
│   ├── Dockerfile
│   └── satusky.toml
└── worker/
    ├── app.py
    ├── requirements.txt
    ├── Dockerfile
    └── satusky.toml

Each service has its own satusky.toml so you can deploy and configure them independently from their own directories.

Dockerfile: Job API

FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY app.py .
EXPOSE 8080
CMD ["python", "app.py"]

api/requirements.txt:

flask==3.0.3
redis==5.0.8

api/app.py:

import os
import uuid
import redis
from flask import Flask, jsonify

app = Flask(__name__)

r = redis.from_url(os.environ["REDIS_URL"])

@app.route("/enqueue", methods=["POST"])
def enqueue():
    job_id = str(uuid.uuid4())
    r.lpush("jobs", job_id)
    return jsonify({"job_id": job_id}), 202

@app.route("/healthz")
def health():
    return "ok"

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=8080)

Dockerfile: Job Worker

FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY app.py .
EXPOSE 8080
CMD ["python", "app.py"]

worker/requirements.txt:

flask==3.0.3
redis==5.0.8

All Satusky deployments require a port in satusky.toml. Workers need a minimal health-check server alongside the processing loop — Satusky uses it to determine whether the pod is ready to receive traffic and to detect crashes.

worker/app.py:

import os
import threading
import redis
from flask import Flask

app = Flask(__name__)

r = redis.from_url(os.environ["REDIS_URL"])

def process_loop():
    print("Worker started, waiting for jobs...")
    while True:
        # BLPOP blocks up to 5s, then loops — allows clean shutdown on SIGTERM
        result = r.blpop("jobs", timeout=5)
        if result is None:
            continue
        _, job_id = result
        job_id = job_id.decode()
        print(f"Processing job {job_id}")
        # do real work here
        print(f"Job complete {job_id}")

@app.route("/healthz")
def health():
    return "ok"

if __name__ == "__main__":
    t = threading.Thread(target=process_loop, daemon=True)
    t.start()
    app.run(host="0.0.0.0", port=8080)

The processing loop runs in a daemon thread. Flask handles the /healthz probe on port 8080. If the process crashes, both threads go down together — Satusky detects the unhealthy pod and restarts it.

satusky.toml files

All configuration lives in a single [app] section. Do not add separate [build], [resources], or [network] sections — they are not valid and will cause an error.

api/satusky.toml:

[app]
  name   = "job-api"
  port   = 8080
  cpu    = "0.25"
  memory = "128Mi"

worker/satusky.toml:

[app]
  name   = "job-worker"
  port   = 8080
  cpu    = "0.25"
  memory = "128Mi"

Step 1: Deploy the API

Run from the api/ directory. The --image flag skips the cloud build step and uses a pre-built image directly. For your real application, omit --image and 1ctl will build from your Dockerfile. The --wait flag blocks until the pod is Running.

cd api/
1ctl deploy --image nginx:alpine --machine compute-main-01 --wait

Expected output:

💡 Using pre-built image: nginx:alpine
Step 2/5: Creating/updating deployment job-api ✓
Step 3/5: Configuring services job-api ✓
Step 4/5: Setting up environment and storage job-api ✓
Step 5/5: Configuring public routing and dependencies job-api ✓
💡 Generated new domain: thirstymonkey-p2ifwfi.satusky.com
✅ 🚀 Deployment for job-api is successful! Your app is live at: https://thirstymonkey-p2ifwfi.satusky.com
Deployment ID: 379c7982-e756-4968-8ad3-1e331f3d6ece
💡 Waiting for deployment to become healthy...
💡 Deployment status: NotReady (0 pct)
✅ Deployment is healthy — pods Running

Note the deployment ID — you’ll see it in restart and destroy output later. The domain is randomly generated and unique to this deployment.

The API will crash immediately if it tries to connect to Redis — REDIS_URL isn’t injected yet. That’s expected. You’ll add the secret in Step 3 and then restart.

Step 2: Deploy the worker

From the worker/ directory:

cd worker/
1ctl deploy --image nginx:alpine --machine compute-main-01 --wait

Expected output:

💡 Using pre-built image: nginx:alpine
Step 2/5: Creating/updating deployment job-worker ✓
Step 3/5: Configuring services job-worker ✓
Step 4/5: Setting up environment and storage job-worker ✓
Step 5/5: Configuring public routing and dependencies job-worker ✓
💡 Generated new domain: cleverpenguin-oq1ckbk.satusky.com
✅ 🚀 Deployment for job-worker is successful! Your app is live at: https://cleverpenguin-oq1ckbk.satusky.com
Deployment ID: 5e5cda03-ab10-47aa-ae0c-6cd8360d564f
💡 Waiting for deployment to become healthy...
💡 Deployment status: NotReady (0 pct)
✅ Deployment is healthy — pods Running

You now have two independent deployments — job-api and job-worker — each with its own domain, deployment ID, and Kubernetes resources.

Step 3: Add the Redis secret to the API

Secrets are scoped per deployment. Even though both services need the same REDIS_URL value, you must run secret create once per service. Satusky creates a separate Kubernetes secret for each deployment (job-api-secrets and job-worker-secrets) — they are never shared.

Run from the api/ directory:

cd api/
1ctl secret create \
  --kv REDIS_URL="rediss://default:[email protected]:6380"

Expected output:

✅ Secret job-api created successfully

secret create is additive — it merges new keys into the existing secret. Any keys already attached to the deployment are left alone.

Step 4: Add the Redis secret to the worker

Run from the worker/ directory:

cd worker/
1ctl secret create \
  --kv REDIS_URL="rediss://default:[email protected]:6380"

Expected output:

✅ Secret job-worker created successfully

At this point there are two separate Kubernetes secrets: job-api-secrets and job-worker-secrets. Each is scoped to its own deployment. Destroying job-worker will delete job-worker-secrets — job-api-secrets is completely unaffected.

Step 5: Restart both services to pick up the secrets

Secrets are injected as environment variables at pod startup. The current pods were launched before the secrets existed, so they don’t have REDIS_URL yet. A rolling restart replaces the pods one by one with new ones that mount the secrets.

cd api/   && 1ctl deploy restart
cd worker/ && 1ctl deploy restart

Each restart prints three lines:

💡 Initiating rolling restart for deployment 379c7982-e756-4968-8ad3-1e331f3d6ece...
✅ Rolling restart initiated. Pods are being replaced one by one.
💡 Use '1ctl deploy status --deployment-id 379c7982-e756-4968-8ad3-1e331f3d6ece' to monitor progress.

deploy restart does NOT update CPU or memory limits — it only replaces the running pods. To change resource allocation you must run a full deploy (see Step 8).

Step 6: Verify both are running

1ctl deploy list

The table shows all your deployments. Look for job-api and job-worker both in completed status:

DEPLOYMENT ID                         HOSTNAMES   TYPE        STATUS     CREATED
────────────────────────────────────  ──────────  ──────────  ─────────  ────────────
379c7982-e756-4968-8ad3-1e331f3d6ece  ...         production  completed  just now
5e5cda03-ab10-47aa-ae0c-6cd8360d564f  ...         production  completed  just now

For programmatic access, the -o json flag is a global flag — it must come before the subcommand:

1ctl -o json deploy list

Extract the app names and statuses:

1ctl -o json deploy list | python3 -c \
  "import sys,json; [print(d['app_label'], d['status']) for d in json.load(sys.stdin)]"

job-api completed
job-worker completed

The JSON fields you’ll use most: app_label, status, deployment_id, cpu_request, memory_request.

Step 7: Check logs for each service

cd api/    && 1ctl logs --tail 3
cd worker/ && 1ctl logs --tail 3

Each command prints a header, the last N lines from the running pod, then a footer:

Pod Logs
────────
[2026-04-28 09:00:59] [job-api-7b68c6c5b5-pvx9q] 2026/04/28 01:00:59 [notice] 1#1: exit
[2026-04-28 09:00:59] [job-api-7b68c6c5b5-pvx9q] 2026/04/28 01:00:59 [notice] 1#1: worker process 35 exited with code 0
[2026-04-28 09:00:59] [job-api-7b68c6c5b5-pvx9q] 2026/04/28 01:00:59 [notice] 1#1: signal 17 (SIGCHLD) received from 35
---
Showing last 3 lines

Each line is prefixed with a timestamp and the pod name in brackets. For a real Python app you would see your application log lines here. To follow logs in real time, use 1ctl logs stream (press Ctrl+C to stop streaming — the service keeps running).

Step 8: Scale the worker — redeploy with new resources

Job volume has grown and the worker needs more CPU. Edit worker/satusky.toml to increase the allocation:

[app]
  name   = "job-worker"
  port   = 8080
  cpu    = "0.5"
  memory = "256Mi"

Then redeploy. A full deploy is required to apply new resource limits — deploy restart only rolls the pods, it does not update CPU or memory.

cd worker/
1ctl deploy --image nginx:alpine --machine compute-main-01

Expected output (no --wait this time, so it returns immediately after Kubernetes accepts the rollout):

💡 Using pre-built image: nginx:alpine
Step 2/5: Creating/updating deployment job-worker ✓
Step 3/5: Configuring services job-worker ✓
Step 4/5: Setting up environment and storage job-worker ✓
Step 5/5: Configuring public routing and dependencies job-worker ✓
✅ 🚀 Deployment for job-worker is successful! Your app is live at: https://cleverpenguin-oq1ckbk.satusky.com
Deployment ID: 5e5cda03-ab10-47aa-ae0c-6cd8360d564f

Notice two things: the domain stays the same (Satusky reuses the existing public route), and the deployment ID stays the same (it’s an update, not a new deployment). The job-api deployment is completely unaffected — it kept serving requests throughout.

Step 9: Pause the worker for maintenance

To take the worker down without losing jobs — Redis holds them in the list while the worker is offline:

cd worker/
1ctl deploy destroy -y

Expected output:

💡 Destroying deployment 5e5cda03-ab10-47aa-ae0c-6cd8360d564f...
✅ Deployment 5e5cda03-ab10-47aa-ae0c-6cd8360d564f destroyed successfully

deploy destroy removes everything that deploy created for this service: the Deployment, Service, public route, ConfigMap, and all Secrets. The job-worker-secrets Kubernetes secret is gone.

The API keeps running. Clients can still POST /enqueue, and job IDs accumulate in the Redis list. This is the key benefit of independent lifecycle management — one service going down doesn’t cascade to the other.

Step 10: Bring the worker back

When you’re ready to resume processing:

cd worker/
1ctl deploy --image nginx:alpine --machine compute-main-01 --wait

Because the previous deployment was destroyed, this creates a completely new deployment with a new deployment ID and a new random domain:

💡 Using pre-built image: nginx:alpine
Step 2/5: Creating/updating deployment job-worker ✓
Step 3/5: Configuring services job-worker ✓
Step 4/5: Setting up environment and storage job-worker ✓
Step 5/5: Configuring public routing and dependencies job-worker ✓
💡 Generated new domain: thirstyowl-aw8sam2.satusky.com
✅ 🚀 Deployment for job-worker is successful! Your app is live at: https://thirstyowl-aw8sam2.satusky.com
Deployment ID: 4e42d51b-412d-4f89-a30f-5e22a6e78229
💡 Waiting for deployment to become healthy...
💡 Deployment status: NotReady (0 pct)
✅ Deployment is healthy — pods Running

The secrets were destroyed along with the old deployment. You must re-add REDIS_URL before the worker can connect to Redis:

1ctl secret create \
  --kv REDIS_URL="rediss://default:[email protected]:6380"
1ctl deploy restart

Output:

✅ Secret job-worker created successfully

💡 Initiating rolling restart for deployment 4e42d51b-412d-4f89-a30f-5e22a6e78229...
✅ Rolling restart initiated. Pods are being replaced one by one.
💡 Use '1ctl deploy status --deployment-id 4e42d51b-412d-4f89-a30f-5e22a6e78229' to monitor progress.

The worker connects to Redis and begins processing the backlog starting from the oldest job.

Step 11: Cleanup

cd api/    && 1ctl deploy destroy -y
cd worker/ && 1ctl deploy destroy -y

After both destroy commands complete, verify that deployment-owned resources are gone:

kubectl -n <namespace> get deploy,svc,httproute,ingress,pvc

Current v1 work should make route and volume cleanup explicit in CLI output instead of relying on a generic success message.

Summary

	job-api	job-worker
Role	Accepts HTTP, enqueues jobs	BLPOPs from Redis, processes jobs
Port	8080	8080 (health-check only)
Secrets	REDIS_URL (in job-api-secrets)	REDIS_URL (in job-worker-secrets)
Scaled independently	Yes	Yes
Can be paused	Not recommended — clients get 503	Yes — Redis holds the backlog
Secrets survive destroy	No — re-add after destroy+redeploy	No — re-add after destroy+redeploy

Key rules to remember

Secrets are per deployment, not per project. Even if two services use the same secret value, you run secret create in each service’s directory separately. The CLI reads satusky.toml in the current directory to know which deployment to attach the secret to.

deploy destroy deletes secrets. There is no way to pause a deployment and keep its secrets. Plan accordingly: keep your secret values noted somewhere safe so you can re-add them after a destroy+redeploy cycle.

Resource changes require a full deploy. deploy restart replaces pods but does not update CPU or memory limits. Change the values in satusky.toml and run deploy --image ... --machine ... again.

-o json is a global flag. It must come before the subcommand: 1ctl -o json deploy list, not 1ctl deploy list -o json.

Two services, one Redis list, full independent lifecycle control. The pattern extends naturally to multiple worker types (email, image processing, payments) — each with its own satusky.toml and its own copy of the secrets it needs.