Migrations

Mental model

golazy.dev/lazymigrate has two sides:

• Source loads ordered migration files.
• Backend records and executes migration steps for a real store.

Migrator sits between them. It loads all sources, asks the backend which IDs are already applied, builds a plan, and calls the backend for each up, down, or redo step.

Migration sources

Application migrations normally live under migrations/<database>/. A small migrations package can embed that tree:

package migrations

import "embed"

//go:embed *
var FS embed.FS

lazymigrate.FromFS(migrations.FS, "postgres") reads direct children from the embedded postgres/ directory, ignores migrations.toml, rejects Go files, and requires each file name to contain a sortable timestamp:

migrations/postgres/202606290001_create_posts.sql
migrations/postgres/202606290002_add_post_status.sql

The file extension does not affect the ID. The first file above has ID 202606290001_create_posts.

Package backends can expose embedded migrations as lazymigrate.Source values:

var catalog lazymigrate.Catalog

_ = catalog.Add("postgres", lazymigrate.FromFS(migrations.FS, "postgres"))
_ = catalog.Add("postgres", pgjobs.Migrations())
_ = catalog.Add("postgres", pgfiles.Migrations())

The catalog is useful when direct package use needs application tables plus package tables. Duplicate IDs across all selected sources are rejected.

Not every migration source comes from an embedded filesystem. For example, golazy.dev/lazyassets/assetmigrate builds one source from the current app asset registry and records completion in object storage. Its migration ID is derived from the build version, asset manifest, and upload options.

Backend contract

A migration backend implements:

type Backend interface {
    Setup(context.Context) error
    List(context.Context) ([]lazymigrate.BackendMigration, error)
    Run(context.Context, lazymigrate.Step) error
    DumpSchema(context.Context) ([]byte, error)
    LoadSchema(context.Context, []byte) error
}

Setup prepares backend metadata, such as a migration table. List returns the applied migration IDs. Run receives a Step with DirectionUp or DirectionDown plus the raw migration content. The backend owns the concrete file format, transactions, locks, metadata writes, and rollback behavior.

Production backends must be safe when several app instances attempt migrations at the same time. The backend must synchronize migration application so only one caller can apply a conflicting step or plan, and the schema change plus migration metadata update must commit atomically. A backend may use a database advisory lock, a transactional row lock, an atomic compare-and-swap record, or another primitive native to its store, but stale concurrent plans must not corrupt the schema or metadata.

DumpSchema and LoadSchema are schema snapshot hooks. A backend can return a clear unsupported error until it grows real dump/load semantics.

Implement a backend

A backend should:

1. Validate that required clients or pools are configured.
2. Create its metadata table or collection in Setup.
3. Return stable, unique applied IDs from List.
4. In Run, parse the backend-specific migration format.
5. Run each step under the store's safest locking and transaction model.
6. Commit the schema change and metadata update as one atomic operation.
7. Record the ID after a successful up step and remove it after a successful down step.
8. Make concurrent callers wait, retry, or observe the already-applied state instead of applying the same step twice.
9. Return errors with enough context to identify the migration ID and phase.

PostgreSQL's backend follows this pattern: it parses -- +lazy Up and -- +lazy Down SQL sections, runs the selected section and metadata update in a transaction, holds an advisory migration lock, and stores applied IDs in lazy_migrations.

Bundle with lazyapp

Conventional applications configure migrations through lazyapp.Config:

func Migrations(ctx context.Context) (lazymigrate.Databases, error) {
    db, ok := pg.FromContext(ctx)
    if !ok {
        return nil, fmt.Errorf("postgres pool missing")
    }
    assetDB, err := assetmigrate.DB(ctx, assetmigrate.Config{
        Storage:       assetStorage,
        Mode:          lazyassets.UnpackBoth,
        WriteManifest: true,
    })
    if err != nil {
        return nil, err
    }
    return lazymigrate.Databases{
        "postgres": {
            Backend: pgmigrate.New(db),
            Sources: []lazymigrate.Source{
                lazymigrate.FromFS(migrations.FS, "postgres"),
                pgjobs.Migrations(),
                pgfiles.Migrations(),
            },
        },
        "assets": assetDB,
    }, nil
}

Sources combines application migrations and package migrations. Each database entry has its own backend, so an app can later migrate PostgreSQL, SQLite, search indexes, object-storage assets, or another store without pretending all backends are the same. Asset upload migrations are additive: they upload the current registry when pending, but down migrations and schema snapshots are not supported because deployed assets are not deleted automatically.

The app binary runs migrations only when lazyapp is asked:

LAZYAPP_MIGRATE=up ./app    # run pending up migrations, then exit
LAZYAPP_MIGRATE=auto ./app  # run pending up migrations, then continue startup

Unset, empty, or off skips automatic migration. If the environment variable is set but no migrations are configured, or every configured database already has an empty plan, migration mode succeeds without applying steps.

When CONTROL_PLANE_ADDR is set to a separate listener, lazyapp starts the real control plane before migrations run. /livez returns OK so process liveness can pass, and /readyz reports that migrations are still running. In auto mode, that listener stays active and later startup stages mount their handlers on the same control plane. If the control plane shares the app listener, migrations leave it alone and the app starts that listener after migrations finish.

Direct use

Build a migrator with the backend and the selected sources:

backend := pgmigrate.New(db)

migrator, err := lazymigrate.New(lazymigrate.Config{
    Backend: backend,
    Sources: catalog.Sources("postgres"),
})
if err != nil {
    return err
}

plan, err := migrator.Up(ctx, 0)
if err != nil {
    return err
}
_ = plan

List reports applied, pending, and missing migrations. PlanUp, PlanDown, and PlanRedo return the steps without touching the backend. Up, Down, and Redo build the plan and apply it.

A migration recorded by the backend but missing from loaded sources blocks execution plans. That prevents down or redo operations from running with an incomplete source history.

App task registration

Generated apps typically keep a small migration config next to the migration files so local tasks and future generators know which backend package and URL environment variable belong to each database:

[postgres]
backend = "golazy.dev/pg/pgmigrate"
url_env = "DATABASE_URL"

That config does not replace Go registration. It is the app-level convention for tooling; the runtime still constructs a real backend with the database client and passes it to lazymigrate.New.

Continue with PostgreSQL for the current PostgreSQL backend and package migrations.