To take the quizzes, you must log in.
Username: student Password: Formulas.1You can view this site in
the language of your choice.
- Answers and marking
Answers and marking
- ⏫
⏬Table of contents
For a part to be valid, you must give a mark and define an answer for it. Also, grading criteria must be specified in order to check the correctness of a student answer.
Decimal point or comma?
Always use the decimal point (.) with the Formulas question. The decimal comma (,) is not accepted or will be interpreted incorrectly! This applies to all calculations as well as the answers given by the students.
For example, enter as 3.14, not 3,14.
Answer types
The Formulas question supports four answer types. Each type will accept a particular set of numbers, operators, functions and possibly algebraic variables. Depending on the purpose of the quiz, some or all of these answer types may be used.
Answer type Description Number The answer can be entered using ordinary or scientific E‑notation, for example 3.14, 6.626e-34. Numeric The answer can include numbers, arithmetic operators + - * / ^(Alt-94 in Windows) or ** and ( ) as well as the constant , for example 5+1/2, 2^9, 3pi. Numerical formula The answer can include everything that includes a numeric answer, plus the following single variable functions: sin(), cos(), tan(), asin(), acos(), atan(), exp(), log(), log10(), sqrt(), abs(), ceil(), floor(). For example, sin(pi/12), 10 log(2). Algebraic formula The answer can include everything that includes a numerical formula, plus algebraic variables. Note: In the Numeric, Numerical formula and Algebraic formula answer types, spaces are interpreted as multiplication. For example, 10 log(2) and 10*log(2) are interpreted the same way.
Notes:
- Students will also need to know these rules in order to input the answers correctly.
- The possible inputs have the following relation: Number ⊆ Numeric ⊆ Numerical formula ⊆ Algebraic formula.
- The answer requires a list of strings for Algebraic formula and a list of numbers for the other answer types.
- ^ in the algebraic formula means "power", not "exclusive or".
- The juxtaposition between numbers and symbols means a multiplication.
- The format check in the quiz interface displays a warning sign when the format is incorrect for the response type. It gives no information on the correctness of the response.
- All symbols are treated as algebraic variable in the answer type of algebraic formula. Hence, you may need to hint students what symbols should be used in the question.
Example
This question has four parts illustrating the four types of answers.
General Question name! The four types of answers Main question Question text! This question has four parts illustrating the four types of answers. Part 1 Part's mark* 1 Answer type Number Answer* pi() Grading criteria* Relative error < 0.01 Part's text Number Give π with at least 1 % accuracy (3.14, 3.14159, etc.): {_0} Part 2 Part's mark* 1 Answer type Numeric Answer* 10 Grading criteria* Relative error < 0.01 Part's text Numeric Give an expression whose value is equal to 10. For example, 2*5, 6 + 4, etc.: {_0} Part 3 Part's mark* 1 Answer type Numerical formula Answer* 2 Grading criteria* Relative error < 0.01 Part's text Numerical formula Give an expression whose value is equal to 2. For example, 1 + 1, sqrt(4), abs(-10+2*4), log10(10^2), ceil(3.7)/sqrt(4), etc.: {_0} Part 4 Part's mark* 1 Local variables a = {-100:100:1}; b = {-100:100:1}; Answer type Algebraic formula Answer* "a^2 - b^2" Grading criteria* Absolute error == 0 Part's text Algebraic formula Give an expression whose value is equal to a^{2} - b^{2}, for example, (a + b)(a - b): {_0}
The Formulas question should look like this:
Scientific E‑notation
Variables can be defined and number answers can be entered in the standard scientific E‑notation. For example:
5e2 (= 5 x 10^{2} = 500)
Example
This question illustrates the use of the scientific E-notation.
General Question name! E-notation Variables Global variables me=9.10938356e-31; Main question Question text! What is the approximate mass of an electron? Part 1 Part's mark* 1 Answer type Number Answer* me Grading criteria* Relative error < 0.001 Unit kg Part's text m_{e} = {_0}{_u}
The Formulas question should look like this:
Correct answer
Depending on the answer type, the answer options will accept an expression that evaluates to either a list of numbers or a list of strings for Algebraic formula. The size of the list will determine how many input boxes for the part. If only one answer is required, you can specify a number or a string instead of a one element list.
For the answer type of Number, Numeric and Numerical formula, a list of numbers or a single number is required. Suppose the variables are defined, each line below is a possible answer:
pi() [sin(pi()/2), cos(pi()/2)] [ans[0], ans[1], ans[2]] ans
For the answer type of Algebraic formula, a list of strings or a single string is required. Suppose the variables are defined, each line below is a possible answer:
"exp(-a x)" " a x^2 + b y^2" ["a sin(x)", "b cos(x)"]
Note that all algebraic variables must be defined in order to be usable in the answer. For the answers above to work, you need to define the following variables:
a = 2; b = 3; x = {-100:100:1}; y = {1:100:1};
Grading criterion
A grading criterion is required to determine the correctness of the student answer. It requires an expression evaluated to a number whose 0 value means false and 1 value means true. Typically, the expression is either the absolute error or the relative error with a tolerance level.
For a question with only one answer, the absolute error is simply the different between the correct answer and the student answer. Hence if the correct answer is 3.2 and the student answer is 3.1, then the absolute error is |3.2-3.1| = 0.1 You may want to limit the range of correct answers, say to 0.05. In this case, you should select Absolute error < 0.05. The relative error is defined as the absolute error divided by the absolute value of correct answer. See _err and _relerr in Grading variables below for more details.
Grading variables
Most of the time, the absolute error or the relative error satisfy the grading criterion. However, sometimes there is a need for other grading criteria.
The scope of Grading variables contains all local variables and the student answers. You can define your own grading criterion with the student answers. The information related to the student answers and correct answers is stored in a set of special variables that start with an underscore as shown below:
Variable name Description _0, _1, _2, ... The student answers. The first answer is _0 corresponding to the answer box {_0} in the part, etc. _a The list of target answers, as defined in the answer field. _r The list of student answers with the same size as _a. The 0^{th} element is the same as _0, etc. _d The list of differences between each element, where _d = diff(_a,_r);. See the Appendix for the function details. _err The absolute error, using the Euclidean norm |a-r|, i.e. _err = sqrt(sum(map("*",_d,_d))); _relerr The relative error, obtained by dividing the absolute error by the norm of the correct answer |a-r|/|a| , i.e. _relerr = _err/sqrt(sum(map("*",_a,_a))); Notes:
- The corresponding input boxes of _0, _1, ... can be specified as {_0}, {_1}, ... in the part's text.
- _relerr is not defined for algebraic answers! So _err should be used instead.
- For non-algebraic answers, the student answer is rescaled towards the unit of the target answer. For example, if the target answer is 2 m and the student answer is 199 cm, then _a is [2] , _r is [1.99] and _0 is 1.99 . It has no effect if no unit is used.
- When there are more than one answer, for example if _a = [100,100]; and _r = [101,102];, then _relerr = sqrt(1*1+2*2)/sqrt(100*100+100*100) ≅ 0.0158
- In this sense, the answer defined in the answer field is only a model (target) answer because it may not be related directly to the correctness of the answer.
Manual grading criteria
Other than true or false, the grading criterion can be any number between 0 (all incorrect) and 1 (all correct). The value 1 means that the student gets the full mark for this part (see Grading scheme below). A fractional value represents the partial correctness of the student answer. Note that values less than 0 are treated as 0 and that values greater than 1 are treated as 1. The following examples illustrate cases where a manual grading criterion is required.
With the Clean and More site themes, the field for manual Grading Criteria is obtained by checking a box on the left hand side of the field.
With the Boost site theme, the field for manual Grading Criteria is obtained by checking a box as shown below:
Multiple correct answers
The possibility for a student to give any correct answer among several correct answers can be allowed by establishing manually the required Grading criteria.
Example
Give a number x that is a multiple of 7 and satisfies 40 < x < 50.
General Question name! Multiples of 7 between 40 and 50 Main question Question text! Give a number x that is a multiple of 7 and satisfies 40 < x < 50: Part 1 Part's mark* 1 Answer type Number Answer* 1 Grading criteria* _0 == 42 || _0 == 49 Part's text {_0}
In Part 1, Answer is specified as 1 (or any other number) to indicate to the system that only one answer is expected. The grading criteria _0 == 42 || _0 == 49 means that the answer is correct if it is equal to 42 or 49.
The Formulas question should look like this:
Example
Give a number x that is a multiple of 7.
General Question name! Any multiple of 7 Main question Question text! Give a number x that is a multiple of 7: Part 1 Part's mark* 1 Answer type Number Answer* 1 Grading criteria* fmod((_0),7) == 0 Part's text {_0}
In Part 1, Answer is specified as 1 (or any other number) to indicate to the system that only one answer is expected. The grading criteria fmod((_0),7) == 0 means that the answer is correct if its modulo 7 is equal to 0, that is, it is a multiple of 7.
The Formulas question should look like this:
Variable criteria
The above example uses only a fixed number, so how can you grade an answer with random variations? Suppose that you now ask questions with a variable range for each question a ≤ x ≤ a + 10. To determine the correctness, you need to check two criteria. The following example illustrates how to do this.
Example
Give a number x that is a multiple of 7 and satisfies a ≤ x ≤ a + 10, where a is equal to 10, 20, 30,... 90.
General Question name! Multiples of 7 between 10 and 100 (Mark = 0 or 1) Variables Random variables a={10:100:10}; Main question Question text! Give a number x that is a multiple of 7 and satisfies a ≤ x ≤ a + 10, where a is equal to 10, 20, 30,... 90: Part 1 Part's mark* 1 Answer type Number Answer* 1 Grading variables criterion1 = _0 % 7 == 0; criterion2 = a <= _0 && _0 <= a+10; Grading criteria* criterion1 && criterion2 Part's text {_0}
In Part 1, Answer is specified as 1 (or any other number) to indicate to the system that only one answer is expected. The grading variable criterion1 = _0 % 7 == 0 checks whether the remainder is 0. Note that criterion1 is equal to 1 if true, or 0 if false. The grading variable criterion2 = a <= _0 && _0 <= a+10 checks whether the response is in the desired range. The grading criteria criterion1 && criterion2 check both criteria.
The Formulas question should look like this:
Example
This example is the same as the previous one, but we want the student to get half a point if he meets one or the other of the two criteria. To do this, simply change the Grading criteria as follows:
Grading criteria* 0.5*criterion1 + 0.5*criterion2
Remember that criterion1 and criterion2 are equal to 1 if true, or 0 if false. Hence, 0.5*criterion1 + 0.5*criterion2 is 0, 0.5 or 1 depending on the student's answer.
The Formulas question should look like this:
Answer dependent on the previous one
The grading variables and grading criterion can be used to mark a student's answer based on the previous one.
Example
In this example, the student must give two answers:
- If both answers are correct, the score is 1.
- If the first answer is correct and the second answer is incorrect, the score is 0.5.
- If the first answer is incorrect but the second is consistent with the first, the score is 0.5.
- If both answers are incorrect, the score is 0.
Answers in one order or another
Multiple criteria can also be used to allow students to enter their answers in one order or another. This is illustrated in the following example.
Example
Use multiple criteria to allow students to give the roots of a quadratic equation in one order or another.
General Question name! Roots of 2x^2 + 3x + 1 = 0 Main question Question text! {#1} Part 1 Part's mark* 1 Local variables root1 = -1; root2 = -0.5; Answer type Number Answer* [1,1] Grading criteria* (_0 == root1 && _1 == root2) || (_0 == root2 && _1 == root1) Placeholder name #1 Part's text Solve the quadratic equation 2x^{2} + 3x + 1 = 0. There are two roots, which are: {_0} and {_1}
In Part 1, Answer is specified as [1,1] (or any other pair of numbers) to indicate to the system that two answers are expected. The grading criteria (_0 == root1 && _1 == root2) || (_0 == root2 && _1 == root1) means that the correct roots can be given as -1 and -0.5, or -0.5 and -1.
The Formulas question should look like this:
Multiple criteria and errors
When Grading criteria is used, the answers are checked by default with zero error. This is not a problem in the previous examples because the answers are round numbers. However, it is also possible to check the answers with errors other than zero. Note that when Grading criteria is used, the calculation of errors must be provided. The following example illustrates the use of Grading criteria together with absolute and relative errors.
Example
Use Grading criteria to allow students to give answers in any order, with absolute and relative errors.
General Question name! Enter two values in any order, with errors Main question Question text! Enter two values as follows: (answers can be given in any order) 4.7 ± 0.2 absolute error, i.e. any value between 4.5 and 4.9 9.6 ± 1% relative error, i.e. any value between 9.504 and 9.696 Part 1 Part's mark* 1 Local variables exact0=4.7; exact1=9.6; Answer type Number Answer* [1,1] Grading variables # 4.7± 0.2 and 9.6 ± 1% absoluteError0=abs(exact0-_0); criterion0=absoluteError0<=0.20000001; absoluteError1=abs(exact1-_1); relativeError1=absoluteError1/exact1; criterion1=relativeError1<=0.010000001; # 9.6 ± 1% and 4.7± 0.2 absoluteError2=abs(exact1-_0); relativeError2=absoluteError2/exact1; criterion2=relativeError2<=0.010000001; absoluteError3=abs(exact0-_1); criterion3=absoluteError3<=0.20000001; Grading criteria* (criterion0 && criterion1) || (criterion2 && criterion3) Part's text The two values are: {_0} and {_1}
The grading variables calculate the absolute and the relative errors, and define the four criteria to be checked. Grading criteria checks these four criteria. In this example, 0000001 is appended to 0.2 and 0.01 in order to account for the computer inaccuracies and make sure that the limit values (4.5, 4.9, 9.504 and 9.696) are accepted answers. This would not be necessary in other cases.
The Formulas question should look like this:
Example
Use Grading criteria to mark a number answer and a multiple choice, such that both the number and the choice must be correct for the student to get a point. If either the number or the choice is incorrect, the student gets no point.
General Question name! Compass navigation graded Variables Global variables VD=["West","East"]; # Variation Direction Main question Question text! Correcting for variation If the compass course is 200° and the true course is 190°, what is the variation? (The correct answer is: Variation = 10° West.) The number must be correct within ±1°. Both the number and the direction must be correct for the student to get a point. Part 1 Part's mark* 1 Local variables Variation=10; VariationDirection=0; Answer type Number Answer* [Variation,VariationDirection] Grading variables absoluteError0=abs(Variation-_0); criterion0=absoluteError0<=1; # Answer must be correct within ±1° criterion1=VariationDirection==_1; Grading criteria* criterion0*criterion1 Part's text Variation = {_0}° {_1:VD}
The Formulas question should look like this:
Mark for different accuracies
Grading criteria can be used to give different marks for the accuracy of the response, for example full mark for a small error and half mark for a larger error. This is illustrated in the following example.
Example
Use Grading criteria to account for the accuracy of the target answer, equal to 100 m. Give full mark for answers with an absolute error less than 1 m, i.e. for any answer between 99 m and 101 m. Give half mark for answers with an absolute error between 1 m and 5 m, i.e. for any answer between 95 m and 99 m, and between 101 m and 105 m.
General Question name! Enter a value close to or equal to 100 m Main question Question text! Enter a value close to or equal to 100 m: A value within ± 1 m, i.e. any value between 99 m and 101 m is worth 1 point. A value within ± 5 m, i.e. any value between 95 m and 105 m is worth 0.5 point. Part 1 Part's mark* 1 Answer type Number Answer* 100 Grading variables case1 = _err < 1.0000001; case2 = 0.5*(_err < 5.0000001); Grading criteria* max(case1,case2) Unit m Part's text {_0}{_u} Extra options [Global] - Deduction for wrong unit (0-1)* 1 [Global] - Basic conversion rules Common SI unit Combined feedback For any correct response Your answer is correct within ± 1 m (1 point) For any partially correct response Your answer is correct within ± 5 m (0.5 point)
In this example, 0000001 is appended to 1 and 5 in order to account for the computer inaccuracies and make sure that the limit values (95, 99, 101 and 105) are accepted answers. This would not be necessary in other cases.
An equivalent accuracy of the answer is used automatically if the answer is entered in a unit other than m. For example, full mark is given for answers between 99000 mm and 101000 mm, half mark is given for answers between 9500 cm and 9900 cm, and between 10100 cm and 10500 cm, etc.
[Global] - Deduction for wrong unit (0-1)* equal to 1 means that the unit must be specified, otherwise the answer is considered incorrect.
The Formulas question should look like this:
Grading scheme
The following is the grading formula used to grade a particular part:
Symbol Description c Correctness. It takes value between 0 and 1. Boolean false is treated as 0 and true is treated as 1. Other values may be possible if manual condition is used (see Grading criteria) u Deduction for wrong unit (see Unit system). In the formula, it always takes value 0 if the unit is correct under Conversion rules m Default mark of the part. r_n Maximum mark fraction of the n-th submission, for adaptive mode only (See Trial mark sequence) f The computed final mark.
- For non-adaptive mode: f = m*c*(1-u)
- For adaptive mode (see Moodle documentation): f = max(r_n*m*c*(1-u))
The maximum is taken over all submissions. From the above formula, even though a student get a low mark in the first attempt, it is still possible for them to get a higher mark in the following attempt.
Core function diff()
This function depends on context variables in addition to the function parameters. It is also the core function to compare the students' response and model answer (see Grading variables). If input X and Y are a list of string, then X[i] and Y[i] are treated as algebraic formulas and all variables in them must be defined before the location of evaluation. For example:
a = 3; x = {1:100}; d = diff(["a x"],["a x^2"]);
Please note that the actual algebraic formulas should be "3 x" and "3 x^2" in the above case. In the above evaluation, the d will take a finite value but not close to zero because the algebraic formula are different. In general, any evaluation failure between two algebraic formula will result in a infinite value INF, so that the expression, say, sum(d) < 0.01 will always be false.
Idea of grading algebraic answer
The evaluation will take place at N randomly selected points defined in all algebraic variables. The result of diff("f(x,y)","g(x,y)") will be the root mean square difference at all evaluation points (1/N)Σ_{i}(f_{i}-g_{i})^{2}, which will converge when N tends to infinity. The N is 100 by default if it is not specified.