Retrieve an audit's question set

Fetch an audit's question set when you need more than the audit's responses[] carry: each question's severity or question_type, the set's set_name and set_code, or the questions that got no response. An audit names its set only by set_id. Join the set to the responses and cache it by set_id, so a sync fetches each set once, however many audits it covers.

Prerequisites

An access token in $BRELLIUM_ACCESS_TOKEN, cached and reused (Authenticate with OAuth2 client credentials).
An audit's set_id, for example from GET /v2/audits?status=complete&limit=1 (Make your first request). This page uses the sample audit a1b2c3d4-e5f6-4a7b-8c9d-0e1f2a3b4c5d, whose set_id is 3c2b1a09-8f7e-4d6c-b5a4-938271605f4e.
With a live key you get your organization's question set, and the audit's evidence can quote the document, which is protected health information (PHI). A sandbox key returns the sample set.

Steps

Fetch the question set named by set_id

set_id is the only link from an audit to its question set; the audit doesn't include the set (Retrieve an audit by id).

curl "https://api.brellium.com/v2/question_sets/3c2b1a09-8f7e-4d6c-b5a4-938271605f4e" \
  --header "Authorization: Bearer $BRELLIUM_ACCESS_TOKEN" \
  --write-out "HTTP %{http_code}\n"

const res = await fetch(
  "https://api.brellium.com/v2/question_sets/3c2b1a09-8f7e-4d6c-b5a4-938271605f4e",
  { headers: { Authorization: `Bearer ${process.env.BRELLIUM_ACCESS_TOKEN}` } },
);
const body = await res.json();
if (!res.ok) throw new Error(`HTTP ${res.status}: ${JSON.stringify(body.error)}`);
console.log(body);
console.log(`HTTP ${res.status}`);

import BrelliumClient from "@brellium/api-client";

const client = new BrelliumClient({ oAuth2: process.env.BRELLIUM_ACCESS_TOKEN });

const questionSet = await client.questionSets.retrieve(
  "3c2b1a09-8f7e-4d6c-b5a4-938271605f4e",
);
console.log(questionSet);

import os

from brellium_api_client import BrelliumClient

client = BrelliumClient(o_auth2=os.environ["BRELLIUM_ACCESS_TOKEN"])

question_set = client.question_sets.retrieve("3c2b1a09-8f7e-4d6c-b5a4-938271605f4e")
print(question_set.to_json())

The response is 200 with the set and all its questions (Retrieve a question set). All nine fields are always present. The response below is the sample set that the sample audit names.

{
  // the audit's set_id
  "set_id": "3c2b1a09-8f7e-4d6c-b5a4-938271605f4e",
  "object": "question_set",
  // v1 called this `name`
  "set_name": "Progress Note Compliance",
  "set_code": "PNC-1",
  // string or null (null for most sets); the `type` list filter matches it exactly
  "type": "treatment_plan",
  "thresholds": [
    // objects, usually { flag_id, score }; older sets can use other fields
    { "flag_id": "5e4d3c2b-1a09-4f8e-9d7c-6b5a49382716", "score": 0.85 }
  ],
  "questions": [
    // the set's questions, in order
    {
      // matches responses[].question_id
      "question_id": "7d6c5b4a-3928-4170-a695-847362514f3e",
      "object": "question",
      "query": "Is the session note signed by the rendering provider?",
      // string or null; "informational" questions don't count toward the score
      "question_type": "default",
      // string or null; organizations can use custom values
      "severity": "Critical",
      // string or null
      "remediation": "Ask the rendering provider to sign the note.",
      // string or null
      "affirmation": null
    },
    {
      "question_id": "6c5b4a39-2817-4069-9584-736251403f2e",
      "object": "question",
      "query": "Does the note document the service start and end times?",
      "question_type": "default",
      "severity": "Normal",
      "remediation": null,
      "affirmation": "Service times are documented."
    },
    {
      "question_id": "5b4a3928-1706-4f58-8473-625140392e1d",
      "object": "question",
      "query": "Which program does this note belong to?",
      "question_type": "informational",
      "severity": "None",
      "remediation": null,
      "affirmation": null
    }
  ],
  // RFC 3339 in UTC
  "created_at": "2026-09-01T12:00:00.000Z",
  "updated_at": "2026-09-24T02:37:12.105Z"
}

The API returns 404 NOT_FOUND (Question set not found) when your organization has no question set with this id; another organization's id and a deleted set return 404 too. An id that is not a UUID returns 400 VALIDATION_ERROR. A new or changed question set can take a few seconds to appear at GET /v2/question_sets/{set_id}; until then you can get 404 or the previous version.

Join responses to questions

Each response in an audit's responses[] already carries its question's remediation and affirmation, so the join adds severity and question_type. Index questions[] by question_id and look up each responses[].question_id. Keep the response when the lookup finds nothing. The lookup misses when the response isn't linked to a question (question_id is null) or when its question was removed from the set or deleted after the audit ran.

# Both objects, joined with jq: severity comes from the set, the rest from the audit
AUDIT=$(curl "https://api.brellium.com/v2/audits/a1b2c3d4-e5f6-4a7b-8c9d-0e1f2a3b4c5d" \
  --silent \
  --header "Authorization: Bearer $BRELLIUM_ACCESS_TOKEN")
SET_ID=$(printf '%s' "$AUDIT" | jq -r '.set_id')
curl "https://api.brellium.com/v2/question_sets/$SET_ID" \
  --silent \
  --header "Authorization: Bearer $BRELLIUM_ACCESS_TOKEN" \
  | jq --argjson audit "$AUDIT" '
      (.questions | map({(.question_id): .}) | add // {}) as $q
      | $audit.responses
      | map({
          answer,
          question,
          severity: ($q[.question_id // ""].severity // null),
          matched: ($q[.question_id // ""] != null)
        })'

// Both objects, joined in code: severity comes from the set, the rest from the
// audit
const headers = { Authorization: `Bearer ${process.env.BRELLIUM_ACCESS_TOKEN}` };

const auditRes = await fetch(
  "https://api.brellium.com/v2/audits/a1b2c3d4-e5f6-4a7b-8c9d-0e1f2a3b4c5d",
  { headers },
);
const audit = await auditRes.json();
if (!auditRes.ok)
  throw new Error(`HTTP ${auditRes.status}: ${JSON.stringify(audit.error)}`);

const setRes = await fetch(
  `https://api.brellium.com/v2/question_sets/${audit.set_id}`,
  { headers },
);
const set = await setRes.json();
if (!setRes.ok)
  throw new Error(`HTTP ${setRes.status}: ${JSON.stringify(set.error)}`);

type Question = { question_id: string; severity: string | null };
type AuditResponse = {
  question_id: string | null;
  question: string;
  answer: string;
};

const byId = new Map<string, Question>(
  (set.questions as Question[]).map((q) => [q.question_id, q]),
);
console.log(
  (audit.responses as AuditResponse[]).map(({ answer, question, question_id }) => {
    const q = question_id ? byId.get(question_id) : undefined;
    return {
      answer,
      question,
      severity: q?.severity ?? null,
      matched: q !== undefined,
    };
  }),
);

import BrelliumClient from "@brellium/api-client";

const client = new BrelliumClient({ oAuth2: process.env.BRELLIUM_ACCESS_TOKEN });

// Both objects, joined in code: severity comes from the set, the rest from the
// audit
const audit = await client.audits.retrieve("a1b2c3d4-e5f6-4a7b-8c9d-0e1f2a3b4c5d");
const set = await client.questionSets.retrieve(audit.set_id);

const byId = new Map(set.questions.map((q) => [q.question_id, q]));
console.log(
  audit.responses.map(({ answer, question, question_id }) => {
    const q = question_id ? byId.get(question_id) : undefined;
    return {
      answer,
      question,
      severity: q?.severity ?? null,
      matched: q !== undefined,
    };
  }),
);

import json
import os

from brellium_api_client import BrelliumClient

client = BrelliumClient(o_auth2=os.environ["BRELLIUM_ACCESS_TOKEN"])

# Both objects, joined in code: severity comes from the set, the rest from the audit
audit = client.audits.retrieve("a1b2c3d4-e5f6-4a7b-8c9d-0e1f2a3b4c5d")
question_set = client.question_sets.retrieve(audit.set_id)

by_id = {q.question_id: q for q in question_set.questions}
rows = []
for r in audit.responses:
    q = by_id.get(r.question_id) if r.question_id else None
    rows.append(
        {
            "answer": r.answer,
            "question": r.question,
            "severity": q.severity if q else None,
            "matched": q is not None,
        }
    )
print(json.dumps(rows, indent=2))

[
  {
    "answer": "No",
    "question": "Is the session note signed by the rendering provider?",
    "severity": "Critical",
    "matched": true
  },
  {
    "answer": "Yes",
    "question": "Does the note document the service start and end times?",
    "severity": "Normal",
    "matched": true
  },
  {
    "answer": "N/A",
    "question": "Which program does this note belong to?",
    "severity": "None",
    "matched": true
  }
]

The full programs below do the same join and cache each set by set_id, as step 3 describes. They print the set's name and code, each answer with its severity, and then the questions that got no response.

// Keep credentials on the server; never in a browser or mobile app.
const token = process.env.BRELLIUM_ACCESS_TOKEN;
if (!token) throw new Error("BRELLIUM_ACCESS_TOKEN is not set");
const BASE_URL = "https://api.brellium.com";

type Question = {
  question_id: string;
  object: "question";
  query: string;
  question_type: string | null;
  severity: string | null;
  remediation: string | null;
  affirmation: string | null;
};
type QuestionSet = {
  set_id: string;
  object: "question_set";
  set_name: string;
  set_code: string;
  type: string | null;
  thresholds: unknown[];
  questions: Question[];
  created_at: string;
  updated_at: string;
};
type AuditResponse = {
  response_id: string;
  question_id: string | null;
  question: string;
  answer: string;
  evidence: string;
  remediation: string | null;
  affirmation: string | null;
};
type Audit = {
  audit_id: string;
  status: string;
  set_id: string;
  responses: AuditResponse[];
};
type ApiError = { error: { code: string; message: string; request_id?: string } };

// Returns null for a 404; throws on any other error.
async function get<T>(path: string): Promise<T | null> {
  const res = await fetch(`${BASE_URL}${path}`, {
    headers: { Authorization: `Bearer ${token}` },
  });
  if (res.status === 404) return null;
  if (!res.ok) {
    const body = (await res.json()) as ApiError;
    throw new Error(
      `${res.status} ${body.error.code}: ${body.error.message} (request_id ` +
        `${body.error.request_id})`,
    );
  }
  return (await res.json()) as T;
}

// One request per set_id for the life of the process. A 404 (the set was deleted)
// is cached as null; a failed request is dropped so the next call retries.
const setCache = new Map<string, Promise<QuestionSet | null>>();
function getQuestionSet(setId: string): Promise<QuestionSet | null> {
  let pending = setCache.get(setId);
  if (!pending) {
    pending = get<QuestionSet>(`/v2/question_sets/${setId}`).catch((err) => {
      setCache.delete(setId);
      throw err;
    });
    setCache.set(setId, pending);
  }
  return pending;
}

// question is null when the response has no matching question in the set.
type JoinedRow = { response: AuditResponse; question: Question | null };

async function joinAudit(audit: Audit): Promise<{
  set: QuestionSet | null;
  rows: JoinedRow[];
  unanswered: Question[];
}> {
  const set = await getQuestionSet(audit.set_id);
  const questions = set?.questions ?? [];
  const byId = new Map(questions.map((q) => [q.question_id, q]));
  const answered = new Set<string>();
  const rows = audit.responses.map((response) => {
    const question = response.question_id
      ? (byId.get(response.question_id) ?? null)
      : null;
    if (question) answered.add(question.question_id);
    return { response, question };
  });
  const unanswered = questions.filter((q) => !answered.has(q.question_id));
  return { set, rows, unanswered };
}

async function main() {
  const audit = await get<Audit>("/v2/audits/a1b2c3d4-e5f6-4a7b-8c9d-0e1f2a3b4c5d");
  if (!audit) throw new Error("No audit with that id in this organization");
  const { set, rows, unanswered } = await joinAudit(audit);
  console.log(
    set
      ? `${set.set_name} (${set.set_code})`
      : `Question set ${audit.set_id} not found; showing responses only`,
  );
  for (const { response, question } of rows) {
    const severity = question?.severity ?? "unknown";
    console.log(`[${severity}] ${response.answer}: ${response.question}`);
  }
  for (const q of unanswered) console.log(`no response: ${q.query}`);
}

main();

# Keep credentials on the server; never in a browser or mobile app.
import os
from functools import lru_cache

import requests

TOKEN = os.environ["BRELLIUM_ACCESS_TOKEN"]
BASE_URL = "https://api.brellium.com"
SESSION = requests.Session()
SESSION.headers["Authorization"] = f"Bearer {TOKEN}"


def get(path: str) -> dict | None:
    """Returns None for a 404; raises on any other error."""
    res = SESSION.get(f"{BASE_URL}{path}", timeout=30)
    if res.status_code == 404:
        return None
    if not res.ok:
        err = res.json()["error"]
        raise RuntimeError(
            f"{res.status_code} {err['code']}: {err['message']} "
            f"(request_id {err.get('request_id')})"
        )
    return res.json()


@lru_cache(maxsize=256)
def get_question_set(set_id: str) -> dict | None:
    """One request per set_id for the life of the process.

    A 404 (the set was deleted) is cached as None; lru_cache does not cache
    a raised exception, so a failed request is retried on the next call.
    """
    return get(f"/v2/question_sets/{set_id}")


def join_audit(audit: dict) -> tuple[dict | None, list[dict], list[dict]]:
    """Each response next to its question (None when the set has no match), plus
    unanswered questions.
    """
    question_set = get_question_set(audit["set_id"])
    questions = question_set["questions"] if question_set else []
    by_id = {q["question_id"]: q for q in questions}
    answered: set[str] = set()
    rows = []
    for response in audit["responses"]:
        question = (
            by_id.get(response["question_id"]) if response["question_id"] else None
        )
        if question:
            answered.add(question["question_id"])
        rows.append({"response": response, "question": question})
    unanswered = [q for q in questions if q["question_id"] not in answered]
    return question_set, rows, unanswered


audit = get("/v2/audits/a1b2c3d4-e5f6-4a7b-8c9d-0e1f2a3b4c5d")
if audit is None:
    raise SystemExit("No audit with that id in this organization")
question_set, rows, unanswered = join_audit(audit)
if question_set:
    print(f"{question_set['set_name']} ({question_set['set_code']})")
else:
    print(f"Question set {audit['set_id']} not found; showing responses only")
for row in rows:
    question = row["question"]
    severity = (question["severity"] if question else None) or "unknown"
    print(
        f"[{severity}] {row['response']['answer']}: {row['response']['question']}"
    )
for q in unanswered:
    print(f"no response: {q['query']}")

Progress Note Compliance (PNC-1)
[Critical] No: Is the session note signed by the rendering provider?
[Normal] Yes: Does the note document the service start and end times?
[None] N/A: Which program does this note belong to?

Show responses[].question next to each answer. It is the wording saved with the response, while questions[].query is the set's current wording, and the two can differ after an edit. A response with no matching question in the set still has its text, answer and evidence; only its severity and question_type are unknown.

Cache each question set once

Every audit scored against the same question set has the same set_id, so a sync over thousands of audits needs each set once. Your organization can send a burst of 100 requests, then 5 per second, shared by every key (Rate limits and errors). Fetching the set once per audit spends that budget on identical responses.

The full programs in step 2 cache by set_id, cache a 404 as "no set" and drop a failed request so the next call retries it. In a long-running service, expire cached sets after an hour. Nothing on the audit tells you its set was edited, so the expiry is the only refresh.

To fetch several question sets in one request, repeat set_id or send the ids in one value separated by commas (Page through and filter lists). A query string over 2,048 bytes is rejected with a bare 403 and no JSON body, so about 46 set_id values fit in one request; split larger batches. The page size still applies, so with more than 20 ids, send limit=100 or follow has_more. The second id below stands for another set_id you have.

curl --get "https://api.brellium.com/v2/question_sets" \
  --data-urlencode "set_id=3c2b1a09-8f7e-4d6c-b5a4-938271605f4e" \
  --data-urlencode "set_id=4d3c2b1a-9e8f-4a7b-8c6d-5e4f3a2b1c0d" \
  --header "Authorization: Bearer $BRELLIUM_ACCESS_TOKEN"

const query = new URLSearchParams();
query.append("set_id", "3c2b1a09-8f7e-4d6c-b5a4-938271605f4e");
query.append("set_id", "4d3c2b1a-9e8f-4a7b-8c6d-5e4f3a2b1c0d");

const res = await fetch(`https://api.brellium.com/v2/question_sets?${query}`, {
  headers: { Authorization: `Bearer ${process.env.BRELLIUM_ACCESS_TOKEN}` },
});
const body = await res.json();
if (!res.ok) throw new Error(`HTTP ${res.status}: ${JSON.stringify(body.error)}`);
console.log(body);

import BrelliumClient from "@brellium/api-client";

const client = new BrelliumClient({ oAuth2: process.env.BRELLIUM_ACCESS_TOKEN });

const questionSets = await client.questionSets.list({
  set_id: [
    "3c2b1a09-8f7e-4d6c-b5a4-938271605f4e",
    "4d3c2b1a-9e8f-4a7b-8c6d-5e4f3a2b1c0d",
  ],
});
console.log(questionSets);

import os

from brellium_api_client import BrelliumClient

client = BrelliumClient(o_auth2=os.environ["BRELLIUM_ACCESS_TOKEN"])

question_sets = client.question_sets.list(
    set_id=[
        "3c2b1a09-8f7e-4d6c-b5a4-938271605f4e",
        "4d3c2b1a-9e8f-4a7b-8c6d-5e4f3a2b1c0d",
    ],
)
print(question_sets.to_json())

{
  "object": "list",
  "data": [
    {
      "set_id": "3c2b1a09-8f7e-4d6c-b5a4-938271605f4e",
      "set_name": "Progress Note Compliance",
      "questions": [ /* ... */ ]
      // ... the other fields as in step 1
    }
  ], // the second set_id matched no set in this organization: left out, no error
  "has_more": false
}

The response is a page of the question sets that matched. An id that matches no question set in your organization is left out, with no error.

List and filter question sets

To fill the cache ahead of time or build a picker, list your organization's question sets (List question sets). Besides the cursor parameters and set_id, the only filter is type, an exact, case-sensitive match. There are no created_* or updated_* filters. Deleted sets are never listed.

curl --get "https://api.brellium.com/v2/question_sets" \
  --data-urlencode "type=treatment_plan" \
  --data-urlencode "limit=100" \
  --data-urlencode "include_total=true" \
  --header "Authorization: Bearer $BRELLIUM_ACCESS_TOKEN"

const query = new URLSearchParams({
  type: "treatment_plan",
  limit: "100",
  include_total: "true",
});

const res = await fetch(`https://api.brellium.com/v2/question_sets?${query}`, {
  headers: { Authorization: `Bearer ${process.env.BRELLIUM_ACCESS_TOKEN}` },
});
const body = await res.json();
if (!res.ok) throw new Error(`HTTP ${res.status}: ${JSON.stringify(body.error)}`);
console.log(body);

import BrelliumClient from "@brellium/api-client";

const client = new BrelliumClient({ oAuth2: process.env.BRELLIUM_ACCESS_TOKEN });

const questionSets = await client.questionSets.list({
  type: "treatment_plan",
  limit: 100,
  include_total: "true",
});
console.log(questionSets);

import os

from brellium_api_client import BrelliumClient

client = BrelliumClient(o_auth2=os.environ["BRELLIUM_ACCESS_TOKEN"])

question_sets = client.question_sets.list(
    type="treatment_plan",
    limit=100,
    include_total="true",
)
print(question_sets.to_json())

{
  "object": "list",
  "data": [
    {
      "set_id": "3c2b1a09-8f7e-4d6c-b5a4-938271605f4e",
      "type": "treatment_plan",
      "questions": [ /* ... */ ]
      // ... the other fields as in step 1
    }
  ],
  "has_more": false,
  "total_count": 1 // present only with include_total=true
}

Values of type include treatment_plan, frequency and coding. Most sets have type: null, which never matches the filter. A value that matches nothing returns "data": [].

Common mistakes

Mistake	What you see	Fix
Fetching the set inside the per-audit loop	`429 RATE_LIMITED` with `Retry-After` once the burst of 100 is spent	Cache by `set_id` (step 3), or fetch question sets ahead of time with the `set_id` or `type` filter.

Retrieve an audit by id

Retrieve one audit, check status before trusting score and flags, handle 404 and 400.

Understand an audit

The audit object, its status lifecycle, flags, updated_at, and its links to other resources.

Page through and filter lists

Cursor pagination, batch get by id, and each resource's filters.

Retrieve an audit by id

Link an audit's people to recipients and users

Retrieve an audit's question set

Prerequisites

Steps

Common mistakes

Next