Package 'ottr'

Description

Execute checks in a test suite and return the TestFileResult object from executing the test. Optionally prints results of the test to console.

Usage

check(test_file, test_env, show_results)

Arguments

Value

The parsed test results for the suite

Examples

## Not run: 
check("tests/q1.R")

## End(Not run)

An R6 class for collecting TestFileResult objects during grading.

Description

A collection of test file results created while grading an assignment

Public fields

Methods

Public methods

• CheckCollector$new()

• CheckCollector$add_result()

• CheckCollector$get_results()

• CheckCollector$clone()

Method new()

Create a CheckCollector. Add a TestFileResult to this collector.

Usage

CheckCollector$new()

Method add_result()

Usage

CheckCollector$add_result(test_file_result)

Arguments

Method get_results()

Usage

CheckCollector$get_results()

Returns

The list of TestFileResult objects

Method clone()

The objects of this class are cloneable with this method.

Usage

CheckCollector$clone(deep = FALSE)

Arguments

Description

An environment into which a collector will be initialized (so we don't need to update global variables).

Usage

collector_env

Format

An object of class environment of length 0.

The name of the active collector variable in collector_env

Description

The name of the active collector variable in collector_env

Usage

collector_varname

Format

An object of class character of length 1.

Description

Execute a string as an R script and return the environment from that execution.

Converts a string to an AST and executes that script in a dummy environment for running test cases against. Transforms all expressions of the form . = ottr::check(...) by replacing the . with an index into a list in the environment with name ⁠check_results_{SECRET}⁠ to collect the TestFileResult objects generated from those checks. (This helps to handle variable name collisions in tests when grading a script.)

Usage

execute_script(script, ignore_errors)

Arguments

Value

The global environment after executing the script

Description

Export a submission to a zip file for submitting. If indicated, a PDF of the submission is generated and included in the zip file. (PDF generation is only supported for Rmd and ipynb files.)

Usage

export(
  submission_path,
  export_path = NULL,
  display_link = TRUE,
  pdf = FALSE,
  force_save = FALSE,
  debug = FALSE
)

Arguments

Examples

## Not run: 
export("hw01.ipynb")

# with pdf
export("hw01.ipynb", pdf = TRUE)

## End(Not run)

Retrieve the global CheckCollector

Description

Retrieve the global CheckCollector

Usage

get_collector()

Description

Execute a script, parse check outputs, and run additional tests specified by the glob pattern tests_glob on the test environment.

Usage

grade_script(script_path, tests_glob, ignore_errors)

Arguments

Value

The GradingResults object after executing tests referenced in the script and those specified by tests_glob

Description

A collection of test case results that correspond to a single test file.

Public fields

Methods

Public methods

• GradingResults$new()

• GradingResults$to_list()

• GradingResults$to_json()

• GradingResults$clone()

Method new()

Create a grading result.

Usage

GradingResults$new(test_file_results)

Arguments

Method to_list()

Convert these results to a JSON-like list that can be convert to a GradingResults object by Otter's Python library.

The returned list has the JSON format

{
  "test_file_results": [
    {
      // output of TestFileResult$to_list
    }
  ]
}

Usage

GradingResults$to_list()

Returns

The generated list

Method to_json()

Export these results to a JSON string.

Usage

GradingResults$to_json()

Returns

The JSON string

Method clone()

The objects of this class are cloneable with this method.

Usage

GradingResults$clone(deep = FALSE)

Arguments

Create a new global CheckCollector

Description

Create a new global CheckCollector

Usage

initialize_collector()

Description

Load test case data from a test file. Executes the file and grabs the global test variable, which should be a list.

Usage

load_test_cases(test_file)

Arguments

Value

The test cases

Description

Run autograder in a Gradescope container and return the results as a properly-formatted JSON string.

Usage

run_autograder(script_path, ignore_errors, test_dir)

Arguments

Value

The JSON string

Examples

## Not run: 
run_autograder("hw01.R", "ABC123", TRUE, "tests")

## End(Not run)

Description

Determine whether this R session is running on Jupyter by checking for a CommManager in IRkernel.

Usage

running_on_jupyter()

Value

A boolean indicating whether IRkernel is running.

Examples

## Not run: 
running_on_jupyter()

## End(Not run)

Description

Attempt to save the notebook by displaying Javascript if running on Jupyter. This function waits until the modification time of the file has changed or until the specified timeout expires.

Usage

save_notebook(nb_path, timeout = 10)

Arguments

Value

A boolean indicating whether the file was saved successfully. If Jupyter is not running, this function returns TRUE.

Examples

## Not run: 
save_notebook("foo.ipynb")

## End(Not run)

Description

A test case for Ottr. Contains configurations and code to be executed for the test.

Public fields

Methods

Public methods

• TestCase$new()

• TestCase$run()

• TestCase$to_list()

• TestCase$clone()

Method new()

Create a test case.

Usage

TestCase$new(
  name,
  code,
  points = 1,
  hidden = FALSE,
  success_message = NA,
  failure_message = NA
)

Arguments

Method run()

Run the test case against the provided environment.

Usage

TestCase$run(env)

Arguments

Method to_list()

Convert this test case to a JSON-compatible list with all of its fields.

Usage

TestCase$to_list()

Returns

The list representation of this test case

Method clone()

The objects of this class are cloneable with this method.

Usage

TestCase$clone(deep = FALSE)

Arguments

Examples

tc = TestCase$new("q1", {
  testthat::assert_true(q1.ans)
})
env = new.env()
env$q1.ans = TRUE
tc$run(env)

Description

Represents the results of running a test case against a global environment. Contains metadata about the passing/failing of the test case as well as a reference to the test case itself.

Public fields

Methods

Public methods

• TestCaseResult$new()

• TestCaseResult$get_score()

• TestCaseResult$repr()

• TestCaseResult$to_list()

• TestCaseResult$get_message()

• TestCaseResult$clone()

Method new()

Create a test case result.

Usage

TestCaseResult$new(passed, error, test_case)

Arguments

Method get_score()

Get the score earned for this test case, accounting for whether the test passed or failed.

Usage

TestCaseResult$get_score()

Returns

The score

Method repr()

Convert this result into a human-readable string for display.

Usage

TestCaseResult$repr()

Returns

The string representation of this result

Method to_list()

Convert this result to a JSON-compatible list with all of its fields.

Usage

TestCaseResult$to_list()

Returns

The list representation of this result

Method get_message()

Get the message to be displayed to the student based on whether the test case passed or failed, if any.

Usage

TestCaseResult$get_message()

Returns

The message or NA

Method clone()

The objects of this class are cloneable with this method.

Usage

TestCaseResult$clone(deep = FALSE)

Arguments

Description

A collection of test case results that correspond to a single test file.

Public fields

Methods

Public methods

• TestFileResult$new()

• TestFileResult$get_basename()

• TestFileResult$get_score()

• TestFileResult$repr()

• TestFileResult$to_list()

• TestFileResult$clone()

Method new()

Create a test file result.

Usage

TestFileResult$new(filename, test_case_results, points = NULL)

Arguments

Method get_basename()

Get the basename of the file this result corresponds to.

Usage

TestFileResult$get_basename()

Returns

The basename of the test file

Method get_score()

Get the total score earned for this test file as a percentage. Uses TestCaseResult$get_score() to determine the points earned for each test case.

Usage

TestFileResult$get_score()

Returns

The score as a percentage.

Method repr()

Convert this result into a human-readable string for display.

Usage

TestFileResult$repr()

Returns

The string representation of this result

Method to_list()

Convert this result to a JSON-compatible list with all of its fields.

Usage

TestFileResult$to_list()

Returns

The list representation of this result

Method clone()

The objects of this class are cloneable with this method.

Usage

TestFileResult$clone(deep = FALSE)

Arguments

Description

Traverse an AST (a list of expressions) and change calls of the form . = ottr::check(...) so that they are appended to a list with name list_name.

If list_name is check_results_XX, then . = ottr::check(...) becomes ⁠check_results_XX[[<int>]] = ottr::check(...)⁠, where ⁠<int>⁠ is an integer

Usage

update_ast_check_calls(tree, list_name)

Arguments

Value

The tree with substitutions made

Description

Determine whether a code snippet has any syntax errors.

Determine whether a code snippet has any syntax errors.

Usage

valid_syntax(script)

valid_syntax(script)

Arguments

Value

Whether the code snippet is valid (can be parsed with parse)

Whether the code snippet is valid (can be parsed with parse)

Examples

s = "
a = TRUE
b = c(1, 2, 3)
d = function(x) x ^ 2
f = d(b)
"
valid_syntax(s)
#> [1] TRUE

s = "
if (TRUE) {
  a = c(1, 2)
"
valid_syntax(s)
#> [1] FALSE
s = "
a = TRUE
b = c(1, 2, 3)
d = function(x) x ^ 2
f = d(b)
"
valid_syntax(s)
#> [1] TRUE

s = "
if (TRUE) {
  a = c(1, 2)
"
valid_syntax(s)
#> [1] FALSE