Create Automated Scoring Questions for Video Interviews
Updated
by
Amy Thomas
Create custom questions in your Interview Content Library to ask your candidates during the video interview.
If the questions you are creating will be asked during a live interview, or you plan to manually evaluate your candidates' video interviews, you will need to create your interview questions in the classic format.
Start the question
Click Manage and select Interview Content Library from the dropdown list. In the Custom Library tab, click Create New Question.
You may add questions in video or text format. If you do not want to record yourself asking a question, you can include written questions for your candidates to respond to instead.
Ensure that Turn Automated Scoring (Beta) on for this question is toggled on.
Step 1: What Competency Will This Question Evaluate?
You can choose an option from the list or add one of your own.
If you add your own competency, this will be validated to make sure it is suitable for automatic scoring. Click Validate after typing this in.
Click Proceed to Next Step.
Step 2: Question Creation
Using a pre-loaded competency
If you have selected one of the preloaded competencies, some example questions are presented. You may select one of these or write your own question.
- If you select an example question, click Proceed to Next Step.
- To add your own question, click Write your own Question.
Type in your question. Please ensure that it aligns with your selected competency and is structured as a behavioral, experience-based, or situational type of question. Click + next to each question type to learn more.
Click Validate. Criteria's AI will check that the question can be automatically scored.
Click Proceed to Next Step.
Using a custom competency
If you have created your own competency, you will need to create your own question as well.
Please structure your question using one of these question types: Behavioral, Experience-based, or Situational. Click the + next to each of these to learn more about these question types.
Click Validate. Criteria's AI will check that the question can be automatically scored.
Click Proceed to Next Step.
Step 3: Approve Response Rating Scales
Using a pre-loaded competency and question
If you have selected one of the preloaded competencies and a suggested question, the rating scale will load immediately.
If you want to make any changes to the Indicators that the AI will score your candidates on, click Edit Rating Scale. Make the required changes, then click Validate. Once this has been validated, click Save Changes.
Tick the box: I Approve this Rating Scale.
Click Approve Rating Scale.
Using a custom question
If you have written your own question (whether you selected a preloaded competency or chose your own), Criteria's AI will need to generate the rating scale. This could take up to a minute to create.
Review the Rating Scale once it has been generated.
If you want to make any changes to the Indicators that the AI will score your candidates on, click Edit Rating Scale. Make the required changes, then click Validate. Once this has been validated, click Save Changes.
Tick the box: I Approve this Rating Scale.
Click Approve Rating Scale.
Step 4: Record Video & Adjust Settings
Complete the required information. Fields that are not required for Automated Scoring or cannot be altered will appear disabled (greyed out).
- Category: Select an available category from the dropdown list or create your own, making your content easier to find in your Interview Content Library.
- To create your own category, click General Questions and start typing in the Enter new category here field. Click Enter to create the new category.
- Response Type: The response type is Video, as the Criteria AI only evaluates video responses.
- Preparation Time: the amount of time candidates have to consider how to respond to the question before the recording begins.
- Answer Time: the amount of time candidates have to respond to the question.
- Record or Upload Video: Here you can record your question directly in the platform or upload a question recorded elsewhere.
Click Save and Complete. You will be returned to your Interview Content Library.
Next steps
Once you have created all the questions and messages you need, you will be ready to create an interview template.
