dc-api-v2

Local development setup

`env.json`

The `env.json` file contains environment variable values for the lambda functions defined in the API for use in local development. You can create an `env.json` file containing the values to run the API against your dev data by running:

make env.json

If the file already exists, it will not be overwritten unless you include `-B` in the make command.

Running the API locally

To start the API in development mode, first make sure you have the correct version of the AWS SAM command line utility installed:

asdf install aws-sam-cli

Then run the following command:

make serve

The API will be available at:

• `https://USER_PREFIX.dev.rdc.library.northwestern.edu:3002`
  • Don't forget to open port 3002 if you want to access it remotely

⚠️ Note the above URLs (which point to your local OpenSearch instance) need _full endpoints_ to resolve. For example:

• `https://USER_PREFIX.dev.rdc.library.northwestern.edu:3002/search`
• `https://USER_PREFIX.dev.rdc.library.northwestern.edu:3002/collections`

View supported endpoints Questions? View the production API documentation

Chaos middleware

The API supports simulated network effects (errors and delays) for local testing via the `CHAOS_CONFIG` environment variable. If the variable is absent the middleware is disabled entirely.

Set it to an inline JSON array:

export CHAOS_CONFIG='[
  { "pattern": "/works/:id", "effect": "error", "status": 500, "chance": 0.3 },
  { "pattern": "/auth/whoami", "effect": "delay", "ms": 500 },
  { "pattern": "/file-sets/*", "effect": "delay", "ms": [100, 800] }
]'

Or set it to the path of a JSON file containing the same array:

export CHAOS_CONFIG=/path/to/chaos.json

Each rule has a `pattern` (matched against the request path) and an `effect`:

All matching rules are evaluated in order. Delay rules accumulate; an error rule short-circuits the request only when it fires — otherwise evaluation continues to the next rule.

Example workflows

Meadow

View and edit information about a specific Work in the Index.

1. Open a local Meadow instance.

2. Find an `id` of a Work you'd like to inspect in the API.

3. View JSON response at `https://USER_PREFIX.dev.rdc.library.northwestern.edu:3002/works/[WORK_ID]`

4. View IIIF Manifest JSON response at `https://USER_PREFIX.dev.rdc.library.northwestern.edu:3002/works/[WORK_ID]?as=iiif`

IIIF content search

IIIF Presentation responses expose IIIF Content Search 2.0 services for transcription annotations:

• Work manifests include a `SearchService2` entry for `https://USER_PREFIX.dev.rdc.library.northwestern.edu:3002/works/[WORK_ID]/search?as=iiif`
• File set canvases include a `SearchService2` entry for `https://USER_PREFIX.dev.rdc.library.northwestern.edu:3002/file-sets/[FILE_SET_ID]/search?as=iiif`

To search transcription text, include a non-empty `q` parameter:

curl "https://USER_PREFIX.dev.rdc.library.northwestern.edu:3002/works/[WORK_ID]/search?as=iiif&q=[QUERY]"
curl "https://USER_PREFIX.dev.rdc.library.northwestern.edu:3002/file-sets/[FILE_SET_ID]/search?as=iiif&q=[QUERY]"

Both endpoints return a IIIF `AnnotationPage` whose `items` target the matching work canvas or file set canvas. Requests without `as=iiif` or a non-empty `q` return `400`.

For help debugging/inspecting, JavaScript `console` messages are written to: `dc-api-v2/dc-api.log`

DC

Develop against changes to the API.

1. Before starting the DC app, temporarily change the port number in `dc-nextjs/server.js` from default `3000` to something like `3003`.

2. Open the port so it can be accessed in the browser.

sgport open all 3003

3. Point to the proxy URL and start DC app (in your `/environment/dc-nextjs` shell)

export NEXT_PUBLIC_DCAPI_ENDPOINT=https://USER_PREFIX.dev.rdc.library.northwestern.edu:3002
bun run dev

Access the app in a browser at: https://USER_PREFIX.dev.rdc.library.northwestern.edu:3003/

Running the API locally with state machine + lambdas (needed for AV download route)

# From the repo root
cd dc-api-v2

# Start the API + step function and associated lambdas
make start-with-step

# Open a second terminal and create the state machine
make state-machine

Deploying a development branch

There are two ways to deploy a development branch: `make deploy` and `make sync`. The differences are:

• Changes: `deploy` deploys a static stack, and requires another `deploy` to update it. `sync` watches for

changes in realtime.

• Dependencies: `deploy` uses the `apiDependencies` resource defined in the template for dependencies, while

`sync` uses the AWS SAM CLI's built-in development dependency logic.

Either way, the resulting stack will be accessible at `https://dcapi-USER_PREFIX.rdc-staging.library.northwestern.edu`.

An existing `sync` stack can be reused by running `make sync` again, or by running `make sync-code` to only

sync code changes (no infrastructure/template changes).

`samconfig.*.yaml`

Both methods involve a `samconfig.USER_PREFIX.yaml` file. This file, with default values, can be created by

running (for example):

make samconfig.mbk.yaml

This will create a configuration to stand up the default stacks in both `deploy` mode (API, AV Download, and Chat) and

`sync` mode (Chat only). To deploy a different combination of features, specify them using the `WITH` option:

make samconfig.mbk.yaml WITH=API,DOCS

Available features are: `API`, `AV_DOWNLOAD`, `CHAT`, and `DOCS`.

⚠️ Be very careful including the API in `sync` mode as every change within `/api` will take a long time to deploy.

As with the `env.json` file, `make` will not overwrite an existing file unless you include `-B`.

Tearing down a development stack

sam delete --stack-name dc-api-USER_PREFIX

Writing Documentation

API documentation is automatically regenerated and deployed on pushes to the staging and production branches. The documentation is in two parts:

Regular Docs

The `docs` directory contains a standard `mkdocs` project, which can be edited using the same tools and format as the main Repository Documentation.

In a nutshell:

1. Clone this project into a working directory (which you probably already have).

2. Edit the Markdown files in the `docs/docs` directory.

3. To run `mkdocs` locally and preview your work:

sgport open all 8000
   make serve-docs

Docs will be accessible at http://USER_PREFIX.dev.rdc.library.northwestern.edu:8000/

OpenAPI/Swagger Docs

We also maintain an OpenAPI Specification under the docs directory in `spec/openapi.yaml`. When `mkdocs` is running, the Swagger UI can be found at http://USER_PREFIX.dev.rdc.library.northwestern.edu:8000/spec/openapi.html. Like the rest of the documentation, changes to the YAML will be immediately visible in the browser.

The existing spec files (`openapi.yaml`, `types.yaml`, and `data-types.yaml`) are the best reference for understanding and updating the spec. It's especially important to understand how `openapi.yaml` uses the `$ref` keyword to refer to reusable elements defined in `types.yaml`, and how `types.yaml` pulls model schemas from `data-types.yaml`.

For an in-depth look, or to learn how to define things for which there aren't good examples in our spec, refer to the full OpenAPI documentation.

Build Artifacts

`openapi.html` renders the Swagger UI directly from the unmodified `openapi.yaml`. In addition, the build process generates a JSON copy of the spec using the OpenAPI Generator CLI. In order to make sure the spec is valid before checking it in, run:

bun run validate-spec

This check is also part of the CI test workflow, so an invalid spec file will cause the branch to fail CI.

DC API Typescript NPM package

Typescript types for the schemas (Works, Collections, FileSets) are automatically published to the nulib/dcapi-types repo on deploys.

• If a deploy to the `deploy/staging` branch contains changes to the `docs/docs/spec/data-types.yaml` file, new types are generated and a commit is made to the `staging` branch of `nulib/dcapi`. This is intended to be for local testing by NUL devs against the private staging API.
• If a deploy to production (`main` branch) contains changes to the `docs/docs/spec/data-types.yaml` file, new types are generated and a PR is opened into the `main` branch of `nulib/dcapi-types`. Also, an issue is created in `nulib/repodev_planning_and_docs` to review the PR and publish the types package (manually).

Versioning

The current API version is maintained in several different project files. To increment the version, use

make version BUMP=

If you don't specify a `BUMP` value, the command will simply print the current version.