Comments (3)
Hi! Paul Pearson here. I'm glad someone is looking into these issues, as they were known issues when I hurriedly wrote the MatrixCheckers.pl macro and the solution I chose was suboptimal. I found that moderate tolerances seemed to work well enough, but evidently I didn't test that thoroughly enough because you've identified issues with the tolerances. I don't have time right now to work on this. I encourage you to work with Nandor, Davide, and others to fix the issues and merge them into the codebase. Thank you for all of your help with this.
from pg.
@paultpearson - Thanks for the comment about the history, etc.
The issue certainly relates to tolerance issues, and since it seems to be more likely to accept incorrect answers (of a hopefully relatively unusual structure) than to reject incorrect ones, I suspect that few students are likely to have noticed or complained.
Further testing with the test problem from the prior message triggers the problem for the answers: (0,1,-1,-1) and (01,1,-1,-1+0.002), so the problem does not really depend on using an ugly multiple of the first basis vector. (I discovered the issue when testing how code was handling such uglier answers.)
After some additional thought on the issue, I think that the root cause of the bug is how the current code checks that the student's vectors are in the space. The existing code:
$CTC = ($C->transpose) * $C;
# S = student, C = correct, X = change of basis matrix
# Solve S = CX for X using (C^T C)^{-1} C^T S = X.
$X = ($CTC->inverse) * (($C->transpose) * $S);
return $S == $C * $X;
calculates a "change of basis matrix $X
" and then checks (in the fuzzy sense) that the student's vectors match $C * $X
. Given this approach, it seems that any vector sufficiently close to a vector in the space will be able to pass the test. Given the relatively large tolerance used in the existing basis_checker_columns()
this is occurring far too easily, in my opinion. It could be that much of the trouble could be avoided if the tolerance, at least for the final comparison, were far smaller.
The proposed code uses "fuzzy" rank calculations with a smaller tolerance to test (in advance) whether each of the student's vectors are in the given space. So far as I can tell, it is far more accurate it detecting vectors which do not belong to the given space.
However, the proposed fuzzy rank code was not sufficiently accurate in detecting linear dependence among the student vectors is some cases, so the proposed code relies on the determinant based test from the existing code as a fall back measure. The fall back code does not seem to be triggered very frequently, but it did catch some things which the rank based code did not. I'm not certain that the fallback code really has much of a practical effect, but I recall one test where without it the number of independent vectors belong to the space was reported incorrectly, as the shift was small enough for the shifted vector to be considered to be in the space while large enough for it to be considered independent of the unshifted vector. This issue led to my decision not to report linear dependence of the submitted answers when some vector in the answer was detected not to be in the space.
Below is a second test problem for a basis of size 3, where the same sort of problem occurs - if the last vector in the answer is shifted a bit (not too much, not too little) from the sum of the first 2 (correct) basis vectors, the incorrect answer will be accepted.
test_CustomBasisChecker2.pg
follows
# Test problem to test/debug issues I had with basis_checker_columns()
# from pg/macros/MatrixCheckers.pl.
DOCUMENT(); # This should be the first executable line in the problem.
loadMacros(
"PGstandard.pl",
"MathObjects.pl",
"parserMultiAnswer.pl",
#"MatrixCheckers.pl",
"rank.pl",
"CustomBasisChecker1.pl", # includes a slightly modified version of basis_checker_columns
# and a redesigned basis_checker_columns_tani which depends on
# the new rank.pl macro
);
TEXT(beginproblem());
$showPartialCorrectAnswers = 1;
$showPartialCredit = 1;
Context('Matrix');
$vec1 = Matrix([[ 1,0,0,1 ]])->transpose;
$vec2 = Matrix([[ 0,1,0,1 ]])->transpose;
$vec3 = Matrix([[ 0,0,1,1 ]])->transpose;
$vec4 = Matrix([[ 1.002,0.999,0,2 ]])->transpose;
$multians1 = MultiAnswer($vec1, $vec2, $vec3)->with(
singleResult => 1,
separator => ',',
tex_separator => ',',
allowBlankAnswers=>0,
checker => ~~&basis_checker_columns_tani,
# checker => ~~&basis_checker_columns, # Had issues fixed by the _tani version
);
Context()->texStrings;
BEGIN_TEXT
Try this problem with both of the options for the checker function.
$PAR
The original checker is expected to ${BBOLD}incorrectly${EBOLD} accept \( $vec1, $vec2, $vec4 \).
$PAR
Find a basis \( \mathcal{B} \) of \( \text{Span}\left\lbrace $vec1, $vec2, $vec3 \right\rbrace \).
$BR
$BCENTER
\( \mathcal{B} = \left\lbrace \;\; \rule{0pt}{50pt} \right. \;\; b_1 = \)
\{ $multians1->ans_array(2) \}
\( , \;\; b_2 = \)
\{ $multians1->ans_array(2) \}
\( , \;\; b_3 = \)
\{ $multians1->ans_array(25) \}
\( \left. \;\; \rule{0pt}{50pt} \;\; \right\rbrace \)
$ECENTER
END_TEXT
Context()->normalStrings;
ANS( $multians1->cmp() );
ENDDOCUMENT(); # This should be the last executable line in the problem.
from pg.
I do hope that I will be able to recruit someone to help test the new code in addition to me before we embark on changing the code in the main pg/macros
library, as that will have very wide consequences as soon a people would upgrade pg
on their servers.
It should be pretty to use CustomBasisChecker1.pl
as a local replacement for MatrixCheckers.pl
in any problem currently using basis_checker_columns
and to swap between using basis_checker_columns
and basis_checker_columns_tani
for testing.
On the other hand, not rolling out a fix does leave the problematic code pretty widely used in a number of OPL linear algebra problems, which I think is not an ideal situation.
@paultpearson - Do you think you might be able to help with looking into this during the summer?
@mgage - Can you think of other people who have worked on the linear algebra material who might be willing to help look into this issue and test the proposed code?
from pg.
Related Issues (20)
- Warnings when using upToConstant where C0 is already defined HOT 1
- Scaffold section with PopUp breaks subsequent MathObjects HOT 2
- parserRadioMultiAnswer default answer checker doesn't handle list objects HOT 1
- Change behaviour of helpLink in hardcopy?
- Currency errors with hardcopy HOT 14
- macros from OPL that moved to pg: issue with reliance on other macro libraries HOT 5
- q within [@...@] PGML blocks HOT 4
- Syntax bug with PGlateximage.pl and negative parameter values HOT 3
- radio buttons with only one option HOT 5
- unicode characters in strings HOT 1
- Variable adjacent to code substitution in TEXT causes hardcopy error. HOT 8
- PGstatisticsmacros error HOT 2
- Inconsistent/incomplete behaviour for radio button custom answer checkers HOT 3
- inserting an answer between two multiAnswer inputs HOT 13
- issue with `q` and parsing in PGML HOT 9
- PGML bold within a word HOT 3
- Macros whose first line starts with =head1 fail to load. HOT 6
- answer hints in multiple choice HOT 13
- CheckboxList reveals the correct answers HOT 3
- MathQuill toolbar placement in a relatively positioned parent - bug when in an RTL course. HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pg.