Foundation
Cheatcodes
Lay of the Land
Code
Version Control
Git & GitHub
Data
Database
Persistence
Search & Indexing
Retrieval
Deployment
Hosting & Environments
CI/CD
Domains
Routing
Auth
Access Control
Operations
Scheduled Jobs
Cron & Pipelines
Workers & Scale
Background Jobs
Interview
System Design
The Case Interview

Level Up

Ten layers of shipping something real

Layers unlocked: 0 / 10
Chapter 0

Cheatcodes for Building

Lay of the land

Every application that works in production — from a dashboard to a trading system to an internal tool — sits on the same set of layers. Miss one and it either doesn't run, doesn't persist, or nobody can reach it

LayerWhat you needExample
ApplicationThe scripts you wrote to solve a pain pointPython, React, Flask
Version ControlTrack changes, collaborate, undo mistakesGit, GitHub, GitLab
DatabaseSave data between sessionsSupabase, Postgres, SQLite
SearchFind things in large datasetsTF-IDF, cosine similarity, pgvector
HostingWhat is running your codeVercel, Dokku, Railway
RoutingDomain, DNS, SSLCloudflare, Squarespace DNS
AuthWho can access whatOAuth, session cookies, JWT
Scheduled JobsCode that runs on a scheduleCron, Celery, Airflow
Workers & ScaleBackground processing and load distributionCelery, RQ, nginx, HAProxy

This is the lay of the land. Each row below is a chapter. Once you understand these layers, you can build and ship anything — the Stack page shows what reusable modules look like on top of this foundation

Packages like requests, flask, supabase, python-pptx, and jinja2 are all tools that sit in these layers. Learning to compose them is how you go from scripts to applications

Chapter 1

Version Control

The save system for code. Every change tracked, every mistake reversible

Save your progress
git add . && git commit -m "initial version"

Git — Local Version Control

  • Commits — save points. Every commit is a checkpoint you can load back to. Broke something? git revert. Want to try a risky approach? Branch, experiment, merge if it works
  • Branches — parallel universes. main is your stable build. Feature branches let you experiment without corrupting your save file
  • Diffs — every change is tracked line-by-line. You can see exactly what changed, when, and why. git diff shows you the delta, git log shows the history

GitHub — Collaboration + Automation

  • Remote repository — your code lives in the cloud. Push your commits, pull others' changes. Everyone works from the same source of truth
  • Pull requests — propose changes, get review, merge. The entire open source ecosystem runs on this loop: fork → branch → commit → PR → review → merge
  • GitHub Actions — automates testing and deployment. Push to main, it runs your tests and deploys. No manual steps. This is CI/CD (continuous integration / continuous deployment)

If you've ever saved before a boss fight, died, and loaded back in — you already understand version control

Chapter 2

Database

Anything that needs to survive a restart needs a database

Insert a row
supabase.table("contacts").insert({"name": "Jane", "email": "j@co.com"})

When to Use What

DatabaseBest forTrade-off
SQLiteLocal apps, prototypes, single-user toolsNo server needed. Lives in a single file. Doesn't scale to concurrent users
PostgreSQLProduction apps, multi-user, complex queriesIndustry standard. Powerful but needs a server (or hosted service)
SupabaseFast setup, built-in auth + REST APIHosted Postgres with a dashboard, row-level security, and real-time. Free tier is generous
FirebaseReal-time apps, mobile-firstGoogle's NoSQL. Great for chat, live updates. Different mental model than SQL
RedisCaching, sessions, rate limitingIn-memory key-value store. Blazing fast but not for permanent storage

Key Concepts

  • Decide early: what's ephemeral (cache, session) vs. what's permanent (users, content, transactions). AI-generated apps skip this and break later
  • Start with hosted Postgres (Supabase, Neon, Railway). You get a REST API, a dashboard, and row-level security without managing a server
  • Migrations — schema changes over time. Your database evolves as your app evolves. Tools like Alembic (Python) or Supabase's built-in migration system track these changes like version control for your data model

Data Feeds as a Layer

Beyond storage, you often need to pull data from external sources — APIs, market feeds, web scraping results. The pattern is the same: define an adapter that normalizes external data into a consistent format your app expects. Think of it as a data access layer — one interface, many backends

Example: a data feed adapter pattern
price = await data_feed.get_tick("AAPL") # same interface, any source

Whether you're pulling stock prices, weather data, or CRM contacts — wrap external APIs in a consistent adapter. Your app code never changes when you swap the data source

Chapter 4

Hosting & Environments

Getting code off your laptop and onto the internet

Environments

Before you deploy, you need isolated environments. Every project gets its own set of packages — break one project's dependencies and the rest don't care

Create a virtual environment and install packages
python -m venv .venv && source .venv/bin/activate && pip install requests
Dimensionvenvconda
What it isolatesPython packages onlyPython packages + system libraries (C, CUDA, etc.)
Python versionUses whichever Python created itCan install and switch Python versions per environment
SpeedLightweight, instant to createHeavier, solves more complex dependency trees
Best forWeb apps, APIs, scripts, most projectsData science, ML, anything needing compiled native libs

Default to venv. Tools like uv make it blazing fast. Reach for conda when you need CUDA drivers or compiled scientific libraries. Never install packages globally

Deploying

Deploy with a git push
git push origin main # CI/CD handles the rest

Frontend vs. Backend

  • Frontend — static sites and SPAs go to Vercel or Netlify. Free tier, instant deploys, CDN-backed. Don't overthink this
  • Backend — APIs and services need a server. Options range from managed platforms (Railway, Render) to self-hosted PaaS (Dokku on a $12/month droplet). CI/CD via GitHub Actions — push to main, it deploys. No manual steps

Containers

A Dockerfile packages your app with all its dependencies into a reproducible image. "Works on my machine" stops being a problem because the container is the machine. Every modern hosting platform runs containers

Minimal Dockerfile
FROM python:3.11-slim
COPY . /app
RUN pip install -r /app/requirements.txt
CMD ["python", "/app/main.py"]
PlatformBest forCost
VercelFrontend, static sites, Next.jsFree tier
RailwayBackend APIs, quick prototypesPay-per-use
RenderFull-stack, background workersFree tier + paid
DokkuSelf-hosted PaaS, full control~$12/mo droplet
Fly.ioEdge deployment, low-latency APIsPay-per-use
Chapter 5

Domains & Routing

How the internet finds your app

  • Domain — a human-readable address (yourapp.com). You buy one from a registrar (Cloudflare, Squarespace, Namecheap). ~$10/year
  • DNS — the phonebook that translates your domain to your server's IP address. An A record points a domain to an IP. A CNAME points one domain to another. Update these in your registrar's dashboard
  • SSL — the padlock icon. Encrypts traffic between users and your server. Free via Let's Encrypt. Most hosting platforms handle this automatically
  • Subdomainsapi.yourapp.com, dashboard.yourapp.com. Each can point to a different service. The reverse proxy (nginx) reads the hostname and routes to the right container

Buy a domain, point it at your server, let the hosting platform handle SSL. That's it. The complexity is in the initial setup, not ongoing maintenance

Chapter 6

Auth

Who can access what

Protect a route with a decorator — one line
@login_required

Authentication vs. Authorization

  • Authentication — who are you? (login, password, OAuth)
  • Authorization — what can you do? (roles, permissions, groups)

Options

ApproachBest forExample
Session-basedTraditional web apps (Flask, Django)Server stores session, browser stores cookie
JWT tokensAPI-first architectures, SPAsStateless, token in header, no server-side session
OAuth"Sign in with Google/GitHub"Delegate auth to a trusted provider
ManagedDon't want to build auth yourselfAuth0, Clerk, Supabase Auth

Decorators

A decorator is a one-line wrapper you put above a function to change its behavior. In Python, it’s the @something syntax. The function still works the same way — the decorator just runs extra logic before (or after) it executes. Think of it as a bouncer at the door: the party inside doesn’t change, but now only the right people get in

  • @login_required — wraps any route. The auth logic runs before your code. No if-statements, no repeated checks. One line locks a page, an API endpoint, or an entire section of your app
  • @role_required("admin") — same pattern, finer control. Only users with the “admin” role get through. Stack decorators to combine rules — @login_required + @group_required("client-x")
  • Start with Supabase Auth or a managed provider unless you have a specific reason to roll your own. Auth demands 100% correctness — a 95% solution still leaks access
Chapter 7

Scheduled Jobs

Code that runs when you’re not watching is where production gets real

The gap between “it works when I run it” and “it runs reliably at 3 AM every day” is the entire discipline of data engineering

Run a script every day at midnight
0 0 * * * /path/to/venv/bin/python /path/to/script.py

When to Use What

ApproachBest forTrade-off
CronSimple recurring tasks — daily reports, hourly data pulls, cleanupZero dependencies. No retry logic, no dependency graph, no visibility when it fails silently
Systemd timersLinux-native scheduling with logging and restart policiesBetter than cron for servers you control. Still no DAG, no cross-job coordination
Celery + BeatPython apps needing background workers with schedulingRetry logic, result backends, priority queues. Needs Redis/RabbitMQ. Operational overhead
AirflowComplex DAGs — multi-step pipelines with dependencies, retries, backfillsIndustry standard for data engineering. Heavy to run. Overkill for less than a pipeline
GitHub ActionsLightweight scheduled jobs without managing infrastructureFree tier generous, built-in logs. Limited to 6-hour runtime, not great for stateful jobs

Decision framework

One script, one schedule → cron or systemd timer. Multiple tasks, no dependencies → Celery Beat or GitHub Actions. Multi-step pipeline where step 3 depends on steps 1 and 2 → Airflow or Prefect. Start with the simplest tool that handles your failure mode. You can always graduate to Airflow later — you can’t easily un-adopt it

Data Engineering — The ETL Pattern

Data pipelines are just scheduled jobs with dependencies and quality gates

  • Extract — pull data from source systems (APIs, databases, files, scraped pages). Write adapters that normalize messy inputs into a consistent schema
  • Transform — clean, deduplicate, enrich, aggregate. This is where domain knowledge lives. A good transform makes downstream consumers simple
  • Load — write to the destination (data warehouse, Postgres, S3). Idempotent writes — running the same job twice shouldn’t corrupt your data
  • Monitor — data quality checks, row count assertions, freshness alerts. The pipeline that fails silently is worse than the one that never runs
ToolBest forTrade-off
Cron + PythonSimple ETL, single data source, personal projectsFastest to build. No orchestration, no backfill, no visibility
AirflowComplex DAGs, team environments, data platform teamsIndustry standard. Heavy to operate. Worth it at scale
PrefectModern alternative to Airflow, better DXPythonic API, cloud-hosted option. Smaller community
dbtSQL-based transforms inside a data warehouseBest-in-class for analytics engineering. Only does the T in ETL
Chapter 8

Workers & Scale

Workers process jobs. Load balancers distribute traffic. Two different problems.

Workers

Background processes that pull jobs from a queue and execute them. For anything too slow for a request/response cycle — sending emails, processing uploads, running ML inference, generating reports

Celery worker example
@celery.task
def generate_report(user_id):
    data = fetch_data(user_id)
    pdf = render_pdf(data)
    send_email(user_id, attachment=pdf)
ToolLanguageBest for
CeleryPythonMost Python web apps. Mature, well-documented, supports scheduling via Beat
Python asyncioPythonConcurrent I/O without threads — API calls, web scraping, DB queries in parallel. Built into Python, no extra dependencies
RQPythonSimpler alternative to Celery. Less config, fewer features
DramatiqPythonModern Celery alternative with better defaults and error handling
BullNode.jsRedis-backed job queue for Node applications
SidekiqRubyThe standard for Rails background jobs
Python async example — fetch 3 APIs concurrently
import asyncio, aiohttp

async def fetch_all():
    async with aiohttp.ClientSession() as session:
        tasks = [session.get(url) for url in urls]
        responses = await asyncio.gather(*tasks)
        return [await r.json() for r in responses]

Workers vs. async: workers handle CPU-bound or long-running jobs in separate processes. Async handles I/O-bound concurrency within a single process — waiting on APIs, databases, or file reads in parallel without blocking. Use both: async inside your app for fast I/O, workers for heavy background jobs

Load Balancing

Distributing incoming requests across multiple instances of your app. For when a single server can’t handle the traffic or you need zero-downtime deploys

  • Workers scale compute — add more workers to process more jobs in parallel. Each worker is independent
  • Load balancers scale traffic — add more app instances behind a balancer to handle more concurrent requests
  • You need workers first — most apps hit background processing limits before they hit traffic limits. A single server with 4 workers handles more real-world load than you’d expect
ToolBest for
nginxReverse proxy + load balancer. The default for most setups
HAProxyHigh-performance TCP/HTTP load balancing
AWS ALBManaged load balancing in AWS. Auto-scaling built in
CloudflareCDN + DDoS protection + basic load balancing
DokkuBuilt-in load balancing for self-hosted PaaS
Chapter 9

System Design Interviews

Approach it like a case interview — structure first, then depth

A system design interview tests whether you can decompose an ambiguous problem into layers, make defensible trade-offs, and communicate your reasoning. The format maps directly to case interviews: you’re given a vague prompt, you structure the problem, you go deep where it matters, and you present back. The domain is different. The muscle is the same

The Framework

Case interview → System design mapping
Clarify the prompt → Scope the requirements
Structure the problem → Identify the layers
Drill into a branch → Deep-dive one component
Synthesize → Summarize trade-offs and next steps
StepWhat to doCase parallelTime
1. ClarifyWho uses it? How many users? Read vs. write heavy? Do the back-of-envelope math — 10M users at 10 requests/day is ~1,200 QPS. That number determines whether you need caching, sharding, or a single Postgres instanceClarifying questions — scope before solving5 min
2. Map the layersDraw high-level architecture: client → API → database → background jobs. Identify which layers the problem actually lives inIssue tree — MECE breakdown into branches5 min
3. Deep-divePick the hardest layer and go deep. In casing the deep-dive is usually a math problem — in system design it’s usually a data modeling or scaling problem. Schema design, query patterns, indexing strategyDrilling into the most impactful branch15 min
4. Trade-offsEvery choice has a cost. SQL vs. NoSQL, sync vs. async, cache vs. real-time, consistency vs. availability. Name what you’re giving up and why. There are always more trade-offs than you think — keep surfacing them“Recommendation is X because Y, trade-off is Z”10 min
5. SynthesizeWalk through the full system end-to-end. What you’d build first (MVP) vs. what scales laterElevator pitch — 30-second summary5 min

Common prompts

PromptCore layersWhere the depth lives
Design TwitterData model, fan-out, feed ranking, real-timeFan-out on write vs. fan-out on read — how does the feed get built?
People search engineSchema, NL parsing, hard/soft constraint rankingHow do you rank results when some requirements are strict and others are preferences?
Job schedulerQueue, dependency DAG, retries, failure alertingHow do you handle a job that depends on two other jobs, one of which failed?
Rate limiterToken bucket, sliding window, distributed stateHow does the limiter work across multiple servers?
URL shortenerHashing, collision handling, redirect latency, analyticsHow do you generate unique short codes at scale without collisions?
Chat systemWebSockets, message persistence, presence, delivery guaranteesWhat happens when a user is offline? How do you guarantee delivery?

Evaluation criteria

  • Structured thinking — decomposed before solving
  • Trade-off awareness — named costs of every decision
  • Depth on the crux — went deep on the hardest part
  • Communication — legible to someone not in the room
  • Prioritization — MVP first, scale later

The structured problem-solving, communication, and orchestration that made you a good consultant will make you the ideal forward deployed engineer

Ready to see what modules look like in practice?

The Stack →