Week 19: What the Student Is Allowed to See

This was a one pull request week, and I want to say so plainly rather than pad it. #8003 opened on the 10th, a results page for a graded
submission, and the rest of the week went on the next piece of the pipeline, which I will get to at the end. The results page looks like
the simplest thing in the autograding project, a score and a table, and it contained the one design decision in the whole pipeline
where a mistake leaks something the instructor explicitly chose to hide.

The problem is not showing the result

A graded submission already has a stored GradingResult. Until this week it existed only in the database, so a student who was graded
had no way to find out. The new route renders the overall score and a table with one row per test case.

The harder half is that some test cases are hidden on purpose. An instructor marks a case hidden so students cannot work out the exact
inputs and outputs the grader expects and write a circuit that satisfies the list instead of the problem. If the page simply printed
every stored value, it would give that away. What a page may print and what the database knows are different sets, and the page is the
only thing standing between them.

Why the stored result is not enough on its own

The breakdown column on a result holds pass or fail for each case, by position, and nothing else. A page built only from it would say
"case 1: passed, case 2: failed" and a student would have no idea what either case tested. Showing the raw JSON had the same problem and
a worse one, since it is unreadable. To show something useful the page needs the input and output pin values, and those live in the
suite, not the result.

This is where last week's design earns its keep. A result stores a snapshot of the suite as it was when the run happened, so the two
halves come from the same moment and their positions line up. If the page read the live suite, an instructor who edited the suite after
grading would shift every row against its result, and a student would see one test's inputs next to another test's verdict. The helper
that joins them depends on that guarantee and says so by only ever reading the snapshot:

def visible_test_groups(result, reveal_hidden)
  snapshot = result.suite_snapshot["groups"] || []
  result.breakdown.each_with_index.flat_map do |group, i|
    hidden = snapshot.dig(i, "hidden") && !reveal_hidden
    Array(group["cases"]).each_with_index.map do |c, j|
      { "label"   => case_label(group["label"], j),
        "passed"  => c["passed"], "hidden" => hidden,
        "inputs"  => hidden ? nil : pin_values(snapshot.dig(i, "inputs"), j),
        "outputs" => hidden ? nil : pin_values(snapshot.dig(i, "outputs"), j) }
    end
  end
end

For a hidden case the pins come back nil, so the view has nothing to print. I put the hiding in the helper rather than in the model on
purpose. GradingResult records what happened and has no opinion about who is looking, and an instructor who opens the same page with
the reveal setting on gets the full table from the same code.

Who can open the page

The action is a member of the project resource, guarded by the same check_view_access that protects the project itself:

before_action :check_view_access, only: %i[show create_fork change_stars grading_result]

I did not add a new permission, and it is worth being careful about why that is safe. The grading page reuses the rule for viewing the
project, so whoever can open the submission can open its result. For an assignment project that is a short list, because a project that
belongs to an assignment is not allowed to be public, which the model validation enforces. The audience is the author, the assignment's
mentors and any collaborators. A new rule would have been a second place for the access logic to drift. The cost of reusing the rule is
that the page's privacy is only as good as the project's. The case that matters most is a hidden test case with the reveal setting off,
and the controller spec checks it in both directions: the pin values are absent from the page when reveal is off and present when it is
on.

The next piece, started

The other thing I did this week was begin the last link in the chain. Once a submission is graded, the instructor's gradebook should
hear about it when the assignment came from an LMS. Autograder::AgsPush takes a project and a score, finds the assignment's LTI
resource link and the author's LTI subject, fetches a token and posts the score to the line item:

def self.call(project:, score:)
  link = project.assignment&.lti_resource_link
  sub = project.author&.lti_sub
  return unless link&.lineitem_url.present? && sub.present?

  token = Lti::AccessToken.fetch(link.lti_deployment, [Lti::Score::SCOPE], signing_key: Lti::KeyManager.private_key)
  Lti::Score.publish(link.lineitem_url, token, user_id: sub, score_given: score.round.clamp(0, 100), score_maximum: 100)
rescue Lti::AccessToken::Error, Lti::Score::Error
  nil
end

It mirrors the existing token and score split so grading code only deals with a project and a score. It also assumes a few things I
had not checked. One of them is stored in a single column, and one of them is a rescue that returns nil. Both of them get questioned next week.

Where the project stands

The student side of the loop exists: a graded submission has a page, the page shows each case, and what an instructor hid stays hidden.
It also depends on two other pull requests, one for the result model and one for the reveal setting, and I opened it before deciding
how to deal with that. The instructor side, pushing the grade out to an LMS, is written and has not met a reviewer.

Next week: the page gets its first review, the LMS side gets its first review, and the same prerequisite comment turns up on three more
pull requests.