Setting up a Marmoset Project (Dropbox)

Project Settings

From the Marmoset course page, click on create new project link.

Enter the following fields:
- project Number: This text field is displayed at the left-most column in the projects list. It serves as the primary identifier to students for which Project to submit to.
- on-time deadline: This field affects the calculation of each student’s grade for the project. Submissions made prior to this deadline fully count towards the student’s grade.
- late deadline: This field affects the calculation of each student’s grade for the project. Submissions made after the on-time deadline and up to this deadline, count with a deduction according to the Late Constant and Late Multiplier fields below. Submissions made after this deadline do not count towards the student’s grade.
- project title: This text field is displayed at the right-most column in the project list. It can be used as a description field for the project, or duplicate the project Number field.
- URL (optional): If used, this field will turn the project Number field into a link with this URL.
- description (optional): This field does not appear to have any effect.
- stack trace policy: This field controls how much information is sent back to students upon a release test. In general, it is recommended to set this to the entire stack trace for Java or full output printed to stdout for C. Then, use the test scripts to control how much feedback is provided rather than depending on Marmoset for this level of control.
- release policy: This field controls when a student can run release tests. It is recommended to set this to after passing all public tests (default). This allows the usage of public tests to serve as smoke tests, checking that the submission meets basic requirements (e.g. successful compilation) before allowing the use of release tests.
- best submission policy: This field affects the calculation of each student’s grade for the project. It determines which of a student’s multiple submission is used for the calculation of their grade.
- # release tests to reveal: This field controls how many failed test results are shown to the student when running a release test. Note that students can see the number of points scored on release tests for passed tests. This setting only controls how much information about the failed tests is revealed when using a release token.
Click the Create Project button.

Test Setup

A test setup is generally packaged as a ZIP file. This package must contain a Makefile and a test.properties file, as well as the test scripts and any support files.

Here is an example test.properties file:

Further examples of minimal setup files can be found in the following git repo:

Create the Test Setup ZIP file

ZIP all your test files into a single ZIP file, including the Makefile and test.properties file.

Do not ZIP the folder containing the Test Setup files. Rather, ZIP the files at the root folder level of the ZIP file. Otherwise, when the build server unzips the Test Setup files, it will be unzipped into a subfolder and the build server will not be able to find the Makefile and test.properties file.
Verify that this ZIP file does NOT include a copy of the canonical submission. As the build server will download the Test Setup files on top of (after) it downloads the student submission into the build folder, the inclusion of the canonical submission in the Test Setup files will result in the student submission being overwritten by the canonical files. This will result in all student submissions passing all tests and being given a free 100% grade.

Within the New test setups section:

Fill in the Comment field (optional). This field can be used to make a short note about the version of the test setup being uploaded.
Click the Choose File button.
Select the ZIP file and click the Open button.
Click the Upload Test Setup button.

Canonical Submission

A canonical submission is a submission that will pass all the tests of the test setup. Marmoset requires the submission of a canonical submission before a test setup can be published. This is a quality control measure that ensures that there exists at least one solution that can pass all tests. In other words, is prevents publishing bad test setups that are impossible to get a perfect score.

A canonical submission is of the same form as expected of the students. It can consist of single file or it can be package of multiple files within a ZIP file.

Within the Canonical solutions section:

Click the Choose File button.
Select the canonical submission file and click the Open button.
Click the Submit project! button.

After both the test setup and the canonical submission have been uploaded, the build servers will run the test setup on the canonical submission. The overall result of this test run will be shown in the Testing setups table. The status field will read tested if the canonical submission passed all tests. Otherwise, it will read failed. If the canonical submission fails testing, either the test setup or the canonical submission needs to be modified and re-uploaded.

Assigning Points

After the canonical submission has been successfully tested, points need to be assigned to each.

From the Testing setups table, click the assign points link for the desired test setup.
Assign an integer number of points for each test. Tests worth zero points are permitted.
Click the Assign points, edit test-setup comment and activate test setup button.

At this point, the test setup is running, but students cannot see it yet.

Make Setup Visible

Students are unable to see the test setup until it is made visible.

In the Project Maintenance → Project Visiblity section, click the make visible button.

ECE Undergrad Labs