Quizzes

A quiz checks the ideas a lab can't: when to reach for one feature over another, what a snippet prints, which setting controls a behavior. Quizzes are practice, not exams. A learner can answer as often as they like, every wrong pick explains itself, and Show the answer appears after the first miss. Labs remain the hands-on proof; quizzes are the quick recall around them.

Like labs, a quiz lives in your lesson markdown, as a fenced ```quiz block of YAML. It is lesson content, so it has everything lessons have: draft vs. published, version history and diffs, repo sync, Library copies and templates. There is no separate question bank to keep in step.

What learners see

Each question answers on its own Check answer:

• A miss shows the why of the option they picked, never which option is right. They can change their answer and check again, as often as they like. A question that says where to review (see:) adds a link to that part of the lesson.
• Show the answer appears after the first miss. It marks the right answer and shows every option's why, so each wrong option gets explained too.
• A right answer shows every option's why the same way.
• Put-in-order questions also say how many lines, from the top, are already in the right place; select all that apply questions say how many right options are still unselected (never which).

Options are shuffled. Each signed-in learner sees a choice or choices question's options in their own order — the same order every time they come back — and visitors share one order. So it doesn't matter where you write the right answer. The answer key, the Quizzes page, the analytics, and the export all use your order (option A is the first one you wrote). When the order carries meaning — a scale, "Both of the above" — add shuffle: false to the question and learners see your order.

Visitors who aren't signed in can answer too; nothing is kept, and a line under the quiz says so. Signed-in learners' answers are saved: a return visit shows the questions they finished, with their answers.

A signed-in learner can also press This question seems wrong and pick why: "The answer marked right is wrong", "My answer should count too", "The question is unclear", or "It's out of date". It's a fixed list — there's no free text to moderate — and every flag lands in Needs attention with its reason.

A complete quiz

```quiz
id: grouping                 # stable id, unique in the lesson
title: Check your understanding   # optional; this is the default
questions:
  - id: where-or-having      # stable id, unique in the quiz
    ask: You want customers whose orders total over €100. Which clause keeps them?
    see: Filtering groups    # a heading of this lesson, reviewed after a miss
    options:
      - text: "`WHERE SUM(amount) > 100`"
        why: WHERE runs before grouping, so there is no sum to compare yet.
      - text: "`HAVING SUM(amount) > 100`"
        correct: true
        why: HAVING filters groups after GROUP BY has built each customer's sum.
      - text: "`ORDER BY SUM(amount) DESC`"
        why: That sorts customers; it doesn't drop any.
```

Writing it in the editor

The editor has one Quiz menu. Choose a question type and Create quiz to insert the first quiz. Its question text is selected: type, press Tab to move through the answers and explanations, Shift+Tab to go back, and Esc when you're done.

Once a lesson has a quiz, the same menu offers Add question. It targets the quiz at your cursor, or the only quiz in the lesson. With several quizzes and the cursor outside them, choose a destination by name; New quiz… in that picker creates another block. The menu stays available while a quiz needs fixing.

Inside a quiz, ⌘⇧Enter (Ctrl+Shift+Enter on Windows and Linux) adds a question of the same shape with the next free id and the same Tab stops. ⌘K → Add question to quiz: … is another keyboard door.

Quiz → Manage questions… opens a compact list. Select a question to edit its Markdown, move it up or down, duplicate it, delete it, or add an answer option. Duplicate questions get fresh ids. Undo in the editor restores these edits. A quiz needs at least one question; remove its whole Markdown block to delete the quiz. Complex YAML lists with aliases or flow syntax remain editable in Markdown.

Question types

A question is choice unless it says otherwise.

Typed answers

  - id: git-status
    type: answer
    ask: Which git command lists the files you've changed?
    accept: ["git status"]   # any of these counts
    why: git status compares your working tree with the last commit.

• accept — the accepted answers. Case, extra spaces, and one pair of wrapping backticks or quotes don't matter (`git status` and GIT STATUS both count); set caseSensitive: true when case matters.
• number with an optional tolerance — for numeric answers: number: 3.14 and tolerance: 0.01 accept 3.13 to 3.15. Learners can type 1 234, 1,234.5, or a decimal comma (3,14); an ambiguous 1,234 counts as either 1234 or 1.234.
• pattern (with optional flags, as in labs) — a regular expression for answers that vary. It's as forgiving as accept: spacing and wrapping backticks never matter, and case doesn't either unless the question sets caseSensitive: true. Add an accept example next to it: that's what Show the answer shows (the editor warns when it's missing).
• why is required: learners read it after a miss and with the answer.

Put in order

  - id: even-loop
    type: order
    ask: Put the lines in order so the loop prints only the even numbers.
    code: |                  # the finished code, one step per line
      for n in range(10):
          if n % 2 == 0:
              print(n)
    decoys: ["    if n % 2 == 1:"]   # optional lines that don't belong
    why: The loop comes first, the test goes inside it, and print runs only when the test passes.

Write the finished code once; the platform cuts it into lines and shuffles them. Each line keeps its indentation. For steps in prose, use items: (a list) instead of code:. Decoys are optional; when a question has some, learners are told not every line belongs. why is required.

Code in questions

Code never goes inside an option's text. It goes in a code: field, written as a | block exactly like a lab's starterCode or solution: every line keeps its indentation, so Python is safe.

• Code in the question — a code: field on a choice, choices, or answer question shows under the question, highlighted ("What does this print?"). Short answers go in the options.
• Code in the options — give an option code: instead of text:. Lines that differ between the options are shaded, so learners compare the differences instead of reading every line. Keep each option short: move the code they share into the question.
• language: — set it once on the quiz (or per question) for highlighting.
• Inline code in ask, text, and why uses backticks as in markdown.

Writing good questions

The editor warns — never blocks — on the classic flaws:

• "All of the above" / "None of the above" — it gives the answer away, or can't be judged alone. Write a real option.
• An option that points at its neighbours ("both of the above", "the option below", "the previous answer", "option B") while the options are shuffled — reword it, or add shuffle: false. Ordinary words like "a price above 100" don't trigger it.
• A hidden NOT — "Which of these is not…" gets misread; write NOT (or EXCEPT) in capitals or bold.
• The right option is by far the longest — test-wise learners pick it without knowing.
• Five or more options — three plausible ones are enough; a fourth and fifth rarely get picked. The question analytics show which ones do.
• Select all that apply, with one right option — use choice.
• A pattern with no accept example — Show the answer would have nothing to show.
• A code option over 12 lines, a repeated option, a see: that matches no heading of the lesson.

And a few habits that make quizzes worth answering:

• Ask one thing the learner will need, not trivia they could look up.
• Write wrong options from real misconceptions — each why then corrects one.
• A question before the explanation works too: learners remember what they were asked about.

see: — where to review

After a miss (and with the answer) a question can link back:

• a ## or ### heading of this lesson, by its text: see: Filtering groups;
• an anchor: see: "#filtering-groups";
• a page of your academy: see: /courses/sql/lessons/joins;
• or an https URL, such as your docs.

Completion and certificates

A lesson completes itself once every question of its quizzes is done (answered right, or the answer shown) and every exercise of its labs has passed. Learners can still finish any lesson with Complete and continue; quizzes never gate anything.

Quizzes count toward a certificate of completion through the lessons they're in. They never count toward an assessed certificate: that one stays earned in labs, by work checked against real execution, which answer dumps and AI can't do for a learner.

Changing a published quiz

Answers are judged by the published quiz, and each answer is saved with the answer key it was judged by:

• Rewording — the question, an option's text, a why, the see: — keeps what learners earned.
• Changing what's right — which option is right, the accepted answers, the number, tolerance, pattern, flags or caseSensitive, the order, adding and removing options, or reordering them so the right one moves (in your order — learners' shuffled order never counts) — starts that question over for everyone from the next publish. Earlier answers stay in the history and the export; the question analytics count the new key only, and say since when.

The draft preview (Preview draft as learner, including the split preview) opens in Try quiz. Answer, retry, reveal, and reorder just as a learner would (options in your order, so the letters match the answer key); Reset answers starts again. Answer key shows every right answer and explanation for proofreading. Switching between the two keeps your trial answers. Preview answers stay in the browser and never affect learner progress or analytics. Autosave refreshes the split preview; use Pause updates while trying it.

The Quizzes page

Admin → Quizzes lists every published quiz of your academy in one table, one row per quiz: its course and lesson, how many questions it has, and how learners answer it. A quiz without a title of its own is listed under its lesson's title, so untitled quizzes don't all read "Check your understanding". Quizzes still live in lessons — the page never edits one. A row opens the quiz's page; Edit in lesson opens the lesson editor with the cursor on the quiz, or on one question from the quiz's page.

Add quiz to a lesson, beside the page title, lets you choose a course and lesson, then opens the editor's Quiz menu on New quiz. Choose the first question type and press Create quiz to insert it. For a repo-managed course, open the lesson to edit its source in GitHub. Draft lessons are available in the picker; their quizzes appear in the list once published.

The course field says Search N courses… and searches every course by title or topic as you type. Your last six choices appear under Recent, remembered in this browser separately for each academy and staff member. Browse all courses opens the full alphabetical list, with draft and published labels. Arrow keys move through the results; Enter selects a course and Escape closes the list. A course selected on the Quizzes page stays selected when you start creating a quiz.

Tab counts name their unit: All quizzes counts quizzes; Needs attention and Recently changed count questions. The short scope line shows whose answers count, with the methodology in an expandable explanation. Back to quizzes restores the tab, search, course, account, sorting, and page you came from, even after changing the account or combined-course view on a quiz's page.

• Search matches quiz titles and ids, lesson and course titles, and every word of a question: its text, code, options, accepted answers, lines to order, and explanations. When the match is inside a question, the row says which one — the quick way to find every question that mentions a flag your product just renamed.
• Course narrows the list to one course's quizzes.
• Answers from narrows every number to one account's learners, or to Individuals (learners with no company account). Every quiz stays listed; only the numbers change. It searches accounts by name, so it works the same with 5 accounts or 5,000.
• Sort by course order (the default), Learners, Right first try (lowest first: the hardest quizzes are the ones worth opening), or Flags.

How each number is counted:

• A quiz is listed while its lesson is published and not deleted, in a course that isn't deleted, a template, or the Lesson Library. A lesson's changes appear when you publish them — the table shows the version learners answer, and marks a lesson whose draft has moved on with Unpublished changes.
• Every number counts answers to each question's current answer key only (Changing a published quiz), and leaves out your team: staff, and learners at accounts marked Internal.
• Learners — people who answered, or showed the answer to, at least one of its questions.
• Finished — of those, people with every question done: answered right, or the answer shown.
• Right first try — of every learner's first answer to each question, the share that was right: one pool across the quiz's questions. Retries are practice and never change it.
• Answers shown — learner-and-question pairs where the learner pressed Show the answer.
• Flags — "This question seems wrong" flags.

A quiz's page is a review workspace: the quiz's numbers beside Answers from (and, for a Library lesson, Numbers from), then the list of its questions beside the one you're looking at.

• The question list shows each question's start, its right-first-try share, and a status: Needs attention, Reviewed, or when it last changed. Sort it by Quiz order, Needs attention first, or Hardest first; ↑ and ↓ move through it. The page opens on the question you followed a link to, else the first that needs attention, else the first question.
• The selected question starts with why it needs attention and the buttons for it — Mark as reviewed, Edit question — then the question as learners see it, then How learners answered: every option with how often learners picked it first, the right answer marked with a check, and the explanation learners read after the most picked wrong answer (Show every option's explanation shows the rest). A typed answer lists what's accepted, with the share of first answers that were right, then the most common wrong answers with how many learners typed each, and the rest as Other wrong answers. A flagged question quotes the reasons learners gave. History and reliability, folded, holds the before/after of its last change and how well it separates strong and weak learners.
• Try the quiz opens the published quiz as learners see it in a side panel, at the question you're on. Answers there grade the same way and are never recorded.

The selected question and the sort are in the page's address, so a refresh, Back, or a shared link reopens the same view. Edit question opens the lesson draft at that question, and Back to quiz review in the editor returns to the same question with the review's filters. On a phone the list is the page; a question opens full width with All questions to go back.

See Dashboard & analytics for the per-question statistics. Finished is labeled Includes answers shown, since completion includes both correct answers and reveals.

Courses with no published quiz, under the table, lists the rest of your courses — not every course needs one; it's there so none is missed. The same table, for one course, sits on that course's analytics page.

Needs attention

The Quizzes page opens on Needs attention whenever a question needs one: a list of questions, most urgent first, each with its reasons and the evidence behind them. A question lands there for any of these reasons:

• Flagged — learners flagged it since it was last reviewed (every flag on its current answer key, when it never was), counted by reason.
• A wrong option is picked more — from 30 first answers, a wrong option was chosen more often than a right one (the right one, for a one-answer question; the least-picked right one, for select-all). It means a wrong answer key — or a common misconception, worth an explanation that answers it.
• Separates backwards — from 30 learners, the question's discrimination is −0.2 or lower: learners who do well on the course's other questions miss this one more often. Check the key and the wording.
• Often given up on — from 30 learners, half or more of the learners who answered it showed the answer.
• Draft restarts answers — the lesson's draft changes what's right, so publishing it will restart this question for everyone who answered.

Each reason is judged on every learner's answers to the question's current answer key, your team left out. The account filter doesn't apply here, since a question's quality doesn't depend on whose answers you read, and more answers judge it better; the search and course filters do.

Every question in the list has Edit in lesson and Mark as reviewed. Reviewing says you've looked and the question stands: the evidence reasons stop counting, and so do the flags you've seen, on the current answer key. A new flag brings the question back; changing its answer key starts over. A draft that restarts answers stays listed — review is about the evidence, not the next publish, so a question listed only for its draft has no Mark as reviewed. Reviewed questions wait under Reviewed questions, each with Reopen.

On a quiz's page with Answers from narrowed to one account, the questions show that account's numbers, and their Needs attention reasons are still judged on every learner — the same verdict as the list. The page shows the same reasons on each question, with the same buttons, and the admin dashboard's Quizzes card names the questions that need attention and links here.

Did my fix work?

When you change a published question — its wording or its right answer — its History and reliability on the quiz's page compares the version before the change with the one since:

• Learners — for each version, the learners whose first answer to it came while it was live. After a rewording (the same answer key), learners who started before the change stay on the "before" side, so "since" is only people who met the new wording first.
• Right first try — the share of those first answers that were right.
• Showed the answer — the share of those learners who then showed the answer to that version.
• Flags — flags raised while that version was live.

Until 30 learners have met the new version, it says the numbers are early. A change made before anyone answered the question has nothing to compare, so it shows no comparison. With Answers from narrowed to one account, the comparison counts that account's learners, like the rest of the question. Moving a question, or editing another one, isn't a change to it; each publish that changes anything in the question's definition — its text, an explanation, the see:, its code, or its right answer — starts a new version.

Recently changed, a tab on the Quizzes page, lists every question changed in the last 90 days after learners had answered it — newest first, each with its right-first-try before → since — so you can follow up on your fixes in one place.

Analytics and data

The Quizzes page and each quiz's page carry the numbers (above). Per question: how many learners answered, how many got it right on the first try, how often each option was picked, how many showed the answer or flagged the question, and — from 30 learners — whether the question separates learners who know the material from those who don't. See Dashboard & analytics.

Every answer, reveal and flag (with its reason) exports as quiz-answers.csv, and the lesson.completed webhook carries a quiz summary — see Integrations & exports.