WISEflow for markers

Annotations are available in FLOWassign, FLOWlock and FLOWmulti type flows, and are a great way to give feedback to the student on a specific part of their submission. For FLOWassign and FLOWlock, annotation tools can be accessed in the lower left of the Assessor tool:

In FLOWmulti type flows, annotations can instead be added from any section header on the exam paper and can additionally be used to allocate points to questions that require manual marking:

For more information on using annotations in WISEflow, you may find our short tutorial video on this useful:

When the assessment period for a flow ends, the flow is automatically archived after one month. Past flows can always be accessed via the Assessor archive at the top right of the flow overview: 

Audio feedback can be delivered through the use of a rubric, if this was enabled when the rubric was built. Assessors can record their audio by clicking the record button. 

The animation below demonstrates how to record feedback and how to playback and listen to it:

Contact a Digital Assessment Adviser for more help and information on creating rubrics.

As an alternative method of delivering audio feedback, you can make your recordings outside of WISEflow and attach the audio file(s) in comments to students instead. 

Where multiple choice questions are in use in FLOWmulti type flows, WISEflow can score these automatically (many other question types can also be set up with automatic scoring). However, human input is still required to finalise the overall grade.

You can use the "Convert points to assessments" feature available in the Assessor tool to rapidly convert FLOWmulti scores for all students into the appropriate final grades. For more information on how to use this, see Convert points to assessments (turning question scores into overall grades). 

We would recommend using Chrome to access WISEflow as this is generally the most stable and supportive browser. The following web browsers are also suggested:

• Edge
• Firefox
• Opera
• Safari

Web browsers not listed here may also be used to access WISEflow, however we cannot guarantee a consistent user experience for them. Make sure that your chosen browser is up to date at all times to prevent errors from occurring when using WISEflow. 

Note: Internet Explorer (IE) is not recommended. Microsoft officially ended support for IE in November 2020 and this browser will likely prevent you from accessing key features on WISEflow and other browser based applications. 

If you find that you are unable to enter or submit any grades in WISEflow, check that you are within the boundaries of the assessment period indicated on the main flow page:

WISEflow will only allow you to enter and submit grades between these dates. 

If you find that you are having difficulty entering or submitting a grade for a specific student, check whether they have an individual assessment deadline. Individual assessment deadlines are indicated in the appropriate column in the list of participants on the main flow page:

As well as being displayed in the mark overview:

Individual assessment deadlines take priority - so if this is set to a date/ time in the past, you will not be able to enter or submit a grade for the student even if the main assessment deadline hasn't ended yet. 

In either scenario, you will need to contact the Taught Programmes Office (TPO) and request an extension to the marking period, either for the entire cohort or for the specific student that is affected. 

If the assessment deadline(s) seem fine, grades may not appear to have submitted correctly in the following scenarios:

• Where double marking is in use (see Double marking for more information).  

• Where the flow has been set up so that final approval of grades is determined by a Reviewer. If this setting is in use intentionally, there is no further action you need to take after submitting your grades, but if you believe this setting should not be in use, clarify this with the module lead and/ or the TPO.

• Where section based marking is in use and the grades will be fully finalised at Manager level (see Section based marking for more information).

Sometimes students may submit work in an unusual file format, particularly when the assignment has been set up on a FLOWhandin type flow (this may also be the case where appendix material has been provided on a FLOWassign flow). 

If you are having difficulty opening a student submission that you have downloaded from WISEflow and this is a video or sound based piece of work, it is recommended that you initially try installing and using VLC Media Player, which is an excellent open source application that will play most video and audio files. 

If this does not resolve the issue or this is a non-video/ audio file, you may need to do some additional troubleshooting on how to open the specific file type that has been submitted. To prevent this issue from occurring in future, you can always direct students to read our Advice on file formats and upload limits in the Students FAQ. 

If you are ready to mark and open a flow only to find that there are no submissions available, this will be because you have not been allocated to any specific students or sections of an exam paper:

In this instance, you should contact the Taught Programmes Office (TPO) who will be able to amend the marking distribution and ensure that you are appropriately allocated. 

If you are expecting to mark work for a module/ assignment and have not had an automated email from WISEflow or cannot see the flow in your overview, contact the Taught Programmes Office (TPO) and they will be able to advise you further. 

As long as your grade has only been entered and not submitted, you can change this as much as you like. You can change grades either via the student dropdown menu in the Assessor tool:

Or by selecting the appropriate grade in the "Assessment" column on the mark overview screen:

Once your grade has been submitted, you can no longer change this unless it is unpicked first. You can ask a user with Manager permissions on the flow to unpick the grade for you (this will usually be a TPO administrator), which will return this to an editable state. 

If you need to change grades after the assessment period has ended, the TPO will additionally need to extend the assessment deadline for you. Depending on how many grades need to change, this could be an extension to the deadline for the entire flow (which will delay the release of grades and feedback to the entire cohort), or this could be for specific students. 

Comments are a great way to deliver overall feedback on a student paper or to summarise the peformance of the entire cohort.

You can make a comment to a specific student paper by selecting this option from the student dropdown menu (which will have the current student's name or ID number):

Or you can make a comment to the entire cohort on the flow by selecting this option from the flow dropdown menu (which will have the title of the flow):

When you add a comment to a flow, this will automatically be added to a list of any comments that you've made. You can access and amend any comments you've made by selecting them from the list:

The comment tool includes:

1. A text editor for written commentary

2. An "Upload" button which allows you to attach a file from either your computer or from Dropbox

3. Sharing/ visibility settings for your comment which determine who will see this information (note: students will not see feedback until the end of the assessment period is reached). 

You may also find our short tutorial video on making comments useful:

 

For more information on other methods of delivering feedback in WISEflow, see Annotation Tools or Using Rubrics.

WISEflow's "Convert points to assessments" feature can be used to rapidly convert and enter grades for students where individual questions have been marked. This is most commonly used on FLOWmulti type flows, whether you have MCQ questions that have been automatically scored, questions that required manual marking, or a combination of the two. 

Note: this feature can only be used in FLOWmulti if you have been assigned to mark specific students. If you have instead been assigned to mark specific sections of the exam paper, see Section Based Marking (FLOWmulti only) for more information.

On a FLOWmulti type flow, each section header will display the following:

1. 1. Points allocated via automatic scoring.

   2. Points that have been allocated manually by the marker via annotations.

   3. The total score allocated to the section.

The combined score for all sections of the exam paper is displayed in the same format in the upper portion of the Assessor tool:

When all of your allocated students have been awarded points (whether automatically or manually) for all questions on the exam, click on the flow dropdown menu and select "Convert points to assessments":

On the mini menu that follows, by default the minimum and maximum scores will be set to the lowest and highest scores achieved by any student who took the exam. Adjust these if necessary so that they correctly reflect the minimum and maximum achievable score of the exam paper:

If appropriate, you can also adjust the grade boundaries on the graph in line with the grade conversion table described in Senate Regulation 2 or 3 for Undergraduate or Postgraduate programmes respectively (there is no need to do this if the flow has been set up to accommodate percentage grades). These changes will update the graph showing how many students are in each grade boundary, as well as the table where the students' calculated grades are shown in blue:

Click "Save distribution settings" to preserve the changes. Once you are satisfied that the distribution and calculated grades are correct, you can click the green "Set final assessment" button to enter all of the grades for the students:

The grades can then be submitted in the usual way (see Entering and submitting grades for more information). 

This feature can also be used where points have been allocated via annotations on FLOWassign and FLOWlock type flows, but this tends to have a much more situational use case. 

Double marking on flows for dissertations or final year projects may be handled differently depending on your department's practice:

Double marking with two Assessors

In some instances, both the first and second marker are assigned as Assessors to the student - in which case WISEflow will expect both parties to enter and submit the same grade.

The image of the mark overview below shows some of the different states for submitted grades that can occur where multiple Assessors are allocated to the same student:

1. 1. Both markers have entered the same grade and this has been successfully submitted.

   2. One marker has submitted a grade but WISEflow is still waiting for the other marker to enter and submit a grade.

   3. Both markers have attempted to submit grades, however these are conflicting values. The markers will need to discuss this and re-enter and submit an agreed grade. Alternatively if the difference between the marks awarded by the first and second marker is 10% or greater, or an agreement cannot be reached, the work should be reallocated to a third marker for further consideration. 

Double marking with one Assessor and one Reviewer

Alternatively, the first marker may be allocated to the student in the Assessor role and the second marker may be added as a Reviewer - in which case some of the marking will take place offline/ outside of WISEflow and the first marker will enter and submit the final agreed grade.

In this scenario, the Reviewer can still use comments or annotations as appropriate to add feedback in WISEflow (consult the module leader for further guidance on your department's specific practices). 

You can download all student submissions to review offline or print by selecting the option to "Download papers as a ZIP file" from the participants list on the main flow page:

When downloading submissions in bulk, you additionally have several options on how to organise the submissions and what to do with the cover sheets completed by students:

Note: if you are going to be printing hard copies of student submissions, it is recommended that you select the option to structure all files in a single folder. 

The ZIP file will take some time to generate, depending on how many student submissions there are and the size of the submissions (i.e. it will take longer to retrieve a ZIP file where there are lots of submissions and/ or these are large files). 

Alternatively, you can download individual student submissions directly by clicking on the appropriate paperclip button in the list of participants: 

Entering grades

Grades can be entered for students via the student dropdown menu in the Assessor tool:

Students that did not hand in any work generally don't need to have a grade entered, as a special "Non-submit" grade will be confirmed for them by the Taught Programmes Office (TPO). 

Note: if a rubric has been attached to the flow, you will need to complete this in order to enter your initial grade - see Using Rubrics for more information. Once done, grades can then be amended from the student dropdown menu as shown above. 

Submitting grades

Grades will need to be submitted in order to finalise them, which will make them visible to the moderator and TPO in the first instance and will also allow them to be released to students later on if appropriate. Grades are submitted via the mark overview.

You can navigate to the mark overview either by clicking "Go to submission of assessments" via the student dropdown menu in the Assessor tool:

Or by clicking the green "Open the mark overview" button on the main flow page:

Note: if the mark overview button is grey instead of green, it means that the flow has been set up so that submission of grades is not required. If you think this has been done in error, consult the module lead and/ or the TPO. 

On the mark overview screen, you will see a list of students and the grades you've entered for them in the "Assessment" column. To submit all entered grades, simply click the green "Submit all" button:

You do not need to have entered grades for all students in order to do this. You can mark a few students, click "Submit all" and then go and mark another batch, etc.

Alternatively, if you only want to submit grades for specific students, you can click on the appropriate value in the "Assessment" column and this will reveal the option to submit the individual grade:

You can also make changes to specific entered grades here without the need to reload the student's submission in the Assessor tool if needed.

If you are completing feedback forms offline/ outside of WISEflow (e.g. by filling out this information on a Word document), you can attach your file in a comment to the student paper - see Comments for more information.

Alternatively, feedback forms can be built into the platform by creating a standard or custom rubric directly in WISEflow. For more information on marking with a rubric, see Using Rubrics. For more information on how to make a rubric, view the Module Leads and Authors FAQ or email WISEflowHelp@brunel.ac.uk to book a training session.

Within the Assessor role, your flow overview will show you a list of all of the current and ongoing flows (assignments) where you've been added in a marking capacity. Progress bars on each flow show you which state the flow is in:

1. 1. The flow is activated but has not yet started. Students are not yet able to upload and submit work to this flow.

   2. Open for participation. In this phase, students will be producing, uploading and submitting their work to WISEflow.

   3. The assessment period has started. You can now access this flow to view the student submissions allocated to you and start delivering feedback, as well as entering and submitting grades.

   4. The flow has ended. At this stage, marking and moderation should be complete. Your delivered feedback is automatically released to the students and the Taught Programmes Office will have additionally released your submitted grades to the students where appropriate. 

The Assessor tool is where marking and the delivery of feedback takes place in WISEflow. As an Assessor, you can open the tool by selecting this option from the main flow page:

1. 1. Exit the Assessor tool and return to the main flow page (as long as you are connected to the internet, WISEflow will have automatically saved all your marking progress). 

   2. The flow dropdown menu. This has the title of the flow on it, and can be used to navigate to the next student submission, to leave general feedback in a comment to the entire cohort of students, or to access the "Convert points to assessments" feature. This can also be used to access statistics on the performance of the cohort in FLOWmulti type flows. 

   3. The student dropdown menu. This has the name, ID number or other identifying information of the student whose work you are currently viewing in the Assessor tool. This menu can be used to leave general feedback in a comment to the current student, enter or overwrite the student's grade, or to access the shared summary which will allow you to view all feedback that has been added by you and other Assessors (if this is shared with you). 

   4. The area which displays the student submission. In FLOWassign, FLOWlock and FLOWmulti type flows, the submitted document can be read and marked directly on the webpage. In FLOWhandin type flows, a list of all the submitted files will be displayed instead. You can still preview appropriate files in the Assessor tool, but many of these may need to be downloaded and viewed outside of WISEflow (feedback and a grade should still be entered via the Assessor tool). 

   5. The total score awarded to the student submission via points. This can be rapidly converted to an overall grade using the "Convert points to assessments" feature. This is used primarily in FLOWmulti type flows (see Convert points to assessments (turning question scores into overall grades) for more information)

   6. The similarity score for the submission you are currently viewing (available in FLOWassign and FLOWlock flow types only). You can click on the button to be taken to the full similarity report.

   7. Appendix material submitted by the student (FLOWassign and FLOWlock only). A number here indicates how many additional files the student has included with their main paper. 

   8. The rubric button. If a rubric has been attached to the flow at Author or Manager level, this button can be clicked to access the rubric and use it for marking. If a rubric has been attached to the flow, you will likely need to complete this for each student before you are permitted to enter and submit a grade. 

   9. The annotations button (FLOWassign, FLOWlock and FLOWmulti only). You can click this to open the last annotation that you made on the student paper, or to access a list view of all annotations made across the entire document. 

   10. Annotation tools (FLOWassign and FLOWlock only). These can be used to annotate the student submission directly, see Annotation Tools for more information. 

As long as the assessment period hasn't ended, you can use WISEflow's offline assessment feature to add grades to a spreadsheet template, which can then be uploaded back into WISEflow for rapid entry of grades. The offline assessment feature can be accessed via the mark overview:

A more detailed guide to using the offline assessment feature can be found here. 

WISEflow was primarily designed to contain all feedback in an online platform, so printing options are limited and the quality of print formatted feedback may vary. Nonetheless, a printable copy of feedback can be produced by opening the shared summary (accessed via the student dropdown menu in the Assessor tool), switching to "Annotation view" and clicking the blue "Print" button:

The default method of marker allocation in FLOWmulti assigns Assessors to students, meaning that you will be able to see and mark the entire exam paper that some (or all) students have submitted. When section based marking is in use, a marker is instead allocated to a specific section of the exam paper, meaning that you will only be able to see and mark part(s) of the exam for an allocated percentage of the cohort. 

If you are an Assessor on a flow using section based marking, you will not enter and submit the overall grade for the student. Instead, you will need to use annotations to award points to a student for each question:

Once you have awarded all the necessary points for a section, click the tick button to approve the section:

The button will turn blue to indicate that marking for the section has been completed and changes can no longer be made to annotations. If you need to make any amendments, you can click the tick again to unlock the section's annotations and edit them - as long as you have not submitted the awarded points. 

Instead of the overall grade, you will see the total approved score for the section(s) of the exam paper you have marked in the student dropdown menu:

The approved score will need to be submitted in the same way as an overall grade. Once you submit your score, this will display with a closed padlock in the mark overview:

You have now completed your marking and the overall grades will need to be confirmed by the module leader or a user with Manager permissions on the flow. As is the case with submitted grades, if you realise that you need to make a change to a submitted score, you will also need to ask a user with Manager permissions to unpick this for you.

For more information on how to unpick or submit final section based scores at Manager level, see Section Based Marking (FLOWmulti only) in the Module Leads and Authors FAQ.

WISEflow uses Turnitin as a text matching service to aid in the detection of potential plagiarism. Student submissions on FLOWassign and FLOWlock type flows are automatically queued for analysis on Turnitin by default. The resulting similarity scores are available for Assessors and Reviewers in the first instance and can be seen in the list of participants on the main flow page:

 

Or on the dedicated button inside the Assessor tool:

Clicking the percentage will open the Turnitin report in a new tab. If you need to learn more about how to use Turnitin, you may visit the Similarity and other related information page.

A student has emailed me to say that they can't see their feedback, what do I do?

In the first instance, double check that the assessment period has definitely ended (including any individual assessment deadline that might have been given for a specific student) and that the visibility/ sharing settings for your annotation, comment or rubric are set so that this is shared with the participant.

If the assessment period has not ended but marking/ moderation has been completed, you may be able to ask the Taught Programmes Office (TPO) to adjust this to an earlier date/ time. If the visibility/ sharing settings are the problem, ensure that you adjust these by ticking "Share with participant":

If neither of these things appear to be a problem, direct the student to get in touch with WISEflowHelp@brunel.ac.uk to explore the problem further. 

A student has emailed me during an exam to say they are having issues uploading their exam script to WISEflow, what do I do?

The exam should ideally have a cover sheet associated with it which contains a specific section on how to contact someone for assistance when there is a technical query (this can vary depending on whether this is an exam or a class test, and whether this is occurring within one of the main examination periods). Direct the student to use the contact details listed on the cover sheet.

If no cover sheet has been provided, tell the student to email WISEflowHelp@brunel.ac.uk quoting the module code in the subject header as a matter of urgency. 

A student has contacted me during an exam, in a panic, to say they have lost all internet connection, what do I do? 

Try to reassure the student and keep them calm, and direct them as above. If the student cannot communicate with anybody effectively online due to their connection problems, they may also be able to call the Examinations hotline on 01895 268860 (9am-5:30pm GMT) as long as this issue is occuring within the official May, August or December/ January examinations period. 

Note: this phone line is unable to provide support outside of the main examinations periods. 

A student has emailed me their exam script and says they couldn't upload it onto WISEflow, what do I do? 

If the student has made contact before the end of the exam, direct them to seek assistance in getting their file(s) uploaded to WISEflow through the proper channels for technical help as listed on the exam's cover sheet. 

If the student has contacted you for assistance after the examination has ended, you will need to direct them to submit a claim for Extenuating Circumstances via eVision. When making their claim, the student should explain in as much detail as possible exactly what went wrong (i.e. exactly what happened when they tried to upload a file? Was there an error message?), as well as providing any evidence they might have and attaching any files that they weren't able to upload.

If the student needs any further information about the Extenuating Circumstances process, the TPO will be able to provide them with additional guidance and advice. 

Using a rubric built into WISEflow is a great way to ensure that feedback is being delivered through a consistent framework and can help speed up the marking process by minimising the need for typing and converting your input into calculated grades. 

Rubrics can be used on all flow types except FLOWmulti. They need to be created by Authors in WISEflow and then attached to a flow by a flow Manager before marking takes place. Please note that markers should NOT add rubrics to the flow even if there is a button that seemingly allows them to add one. Markers adding rubrics to the flow via the Assessor tool has caused errors on past flows therefore, please request your TPO to add the relevant rubric.

Once added to a flow (by a manager), the rubric can be accessed by clicking the "Rubric" button found underneath the student dropdown menu:

Whether you are marking with a standard rubric or a custom rubric, the process is the same: simply fill the rubric out and approve it to complete the process:

You may also find our short tutorial video on using a rubric useful:

 

Unlike other feedback delivery methods, rubrics are linked directly to the entry and submission of grades. This means that:

• Rubrics can only be used by Assessors (Reviewers can still use comments and annotations as appropriate)

• Once a grade has been submitted, the rubric cannot be amended unless the grade is unpicked first (see Changing a grade for more information)

• Rubrics cannot currently be used on formative flows where the entry and submission of grades is not required (as a workaround, placeholder grades could be entered and submitted without necessarily releasing them)

For more information on how to make and attach a rubric, view the Module Leads and Authors FAQ or email WISEflowHelp@brunel.ac.uk to book a training session.