Comments (21)
filesystem looks okay:
Filesystem Size Used Avail Use% Mounted on
/dev/xvda1 7.8G 2.9G 4.5G 39% /
proc 0 0 0 - /proc
sysfs 0 0 0 - /sys
none 4.0K 0 4.0K 0% /sys/fs/cgroup
none 0 0 0 - /sys/fs/fuse/connections
none 0 0 0 - /sys/kernel/debug
none 0 0 0 - /sys/kernel/security
udev 3.7G 8.0K 3.7G 1% /dev
devpts 0 0 0 - /dev/pts
tmpfs 752M 208K 752M 1% /run
none 5.0M 0 5.0M 0% /run/lock
none 3.7G 0 3.7G 0% /run/shm
none 100M 0 100M 0% /run/user
none 0 0 0 - /sys/fs/pstore
systemd 0 0 0 - /sys/fs/cgroup/systemd
/dev/xvdb 30G 45M 28G 1% /mnt
this from TA Mr. Jonze
mr-jonze [8:35 AM]
this is the app i am submitting passes legacy-hw1 but gives fatal error for legacy-hw2 - you can use as a reference if you like --> aqueous-inlet-36299.herokuapp.com admin 0RHterL
[8:36]
it seems intermittent has been reported several times over the past few days - some came back to the forum to say their went through tho
from rag.
mr-jonze has sent me his version of the software with some notes:
lemme know when you have it and ill destroy
[8:56]
looking at the diff now i actually broke the edit.js path - not sure if that is relevant
[8:56]
sry the new.js path
[8:58]
and im not sure what that deal is with the files like "public/javascripts/ckeditor/config.bak" these got modified somehow so i blacklisted them
[9:00]
fix the error i made on line 11 of categories controller if you think the js path maybe important https://github.com/bill-auger/legacy-hw1/commit/346d41dc37736e32e6d82bf0946bbdf6a758447e#diff-eb471b0b3c7c135b4709a71408d7bfabL11
[9:00]
it scored 100% tho
in order to investigate further
from rag.
mr-jonze code grades fine on Legacy HW1, gets the following:
Your submission was recorded at 2016-02-01 09:47:56 +0000 : submission is on time.
Score: 0/100
There was a fatal error with your submission. It either timed out or caused an exception.
on Legacy HW2, while the grader log has the following:
1) Typo should respond to a simple request [0 points]
Failure/Error: page = agent.get($host)
SocketError:
getaddrinfo: Name or service not known
which I've seen on my own checks, but not leading to that timeout error ...
from rag.
I wonder if it is timeouts related to heroku apps sleeping ...
from rag.
I just got the same error from my shell app:
Your submission was recorded at 2016-02-01 10:00:37 +0000 : submission is on time.
Score: 0/100
There was a fatal error with your submission. It either timed out or caused an exception.
frustratingly the log output is minimal, while the output in the tmux window is more extensive, but scrolling is a real pain:
5) The article merge feature should only show the merge button to administrators [15 points]
Failure/Error: page.body.should include 'Merge Articles'
expected "<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Transitional//EN\"\n \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd\">\n<html xmlns=\"http://www.w3.org/19
99/xhtml\">\n<head>\n <meta http-equiv=\"Content-type\" content=\"text/html; charset=utf-8\" />\n <title>c9-test-1 \xE2\x80\x93 content</title>\n <meta http-equiv=\"imagetoolbar\
" content=\"no\" />\n <link href=\"/stylesheets/growler.css?1453218056\" media=\"screen\" rel=\"stylesheet\" type=\"text/css\" />\n<link href=\"/stylesheets/bootstrap.css?145321805
6\" media=\"screen\" rel=\"stylesheet\" type=\"text/css\" />\n<link href=\"/stylesheets/administration_structure.css?1453218056\" media=\"screen\" rel=\"stylesheet\" type=\"text/css
\" />\n<link href=\"/stylesheets/lightbox.css?1453218056\" media=\"screen\" rel=\"stylesheet\" type=\"text/css\" />\n <script src=\"/javascripts/prototype.js?1453218056\" type=\"te
xt/javascript\"></script>\n<script src=\"/javascripts/effects.js?1453218056\" type=\"text/javascript\"></script>\n<script src=\"/javascripts/builder.js?1453218056\" type=\"text/java
script\"></script>\n<script src=\"/javascripts/dragdrop.js?1453218056\" type=\"text/javascript\"></script>\n<script src=\"/javascripts/controls.js?1453218056\" type=\"text/javascrip
t\"></script>\n<script src=\"/javascripts/typo.js?1453218056\" type=\"text/javascript\"></script>\n<script src=\"/javascripts/lightbox.js?1453218056\" type=\"text/javascript\"></scr
ipt>\n<script src=\"/javascripts/growler.js?1453218056\" type=\"text/javascript\"></script>\n<script src=\"/javascripts/typo_carousel.js?1453218056\" type=\"text/javascript\"></scri
pt>\n<script src=\"/javascripts/administration.js?1453218056\" type=\"text/javascript\"></script>\n<script src=\"/javascripts/ckeditor/ckeditor.js?1453218056\" type=\"text/javascrip
t\"></script>\n <script src=\"/javascripts/calendar_date_select/calendar_date_select.js?1453218056\" type=\"text/javascript\"></script>\n<link href=\"/stylesheets/calendar_date_sel
ect/default.css?1453218056\" media=\"screen\" rel=\"stylesheet\" type=\"text/css\" />\n\n <style type=\"text/css\">\n #carousel-content .slide { width: 125; }\n </style>\n</hea
d>\n<body>\n <div class='topbar'>\n <div class='topbar-inner'>\n <div class='container-fluid'>\n <a href=\"https://ancient-bastion-6361.herokuapp.com\" class=\"brand
\">c9-test-1</a>\n <ul class=\"nav secondary-nav\">\n <li class=\"dropdown\" onmouseover='$(\"user_menu\").style.display = \"block\"' onmouseout='$(\"user_menu\").st
yle.display = \"none\"'>\n <a class=\"dropdown-toggle\" href=\"#\">Logged in as </a>\n <ul class=\"dropdown-menu\" id=\"user_menu\" style='display: none'>\n
...
+ <span class='help-block'>Excerpts are posts summaries that are shown on your blog homepage only but won???t appear on the post itself</span>
+ </div>
+ </div>
+
+ <div class=''>
+ <h4>Uploads</h4>
+ <p class='help-block'>Uploads will be displayed as attachments in your RSS feed, but won't appear in your articles.</p>
+ <ul id='attachments' class='inputs-list'>
+ <li id="attachment_1">
+ <input id="attachments_filename_1" name="attachments[filename_1]" type="file" /> <a href="#" onclick="if (confirm(_('Really delete attachment') +' 1?')){
+Effect.toggle("attachment_1",'appear',{afterFinish:function(obj){Element.remove(obj.element);}});
+}; return false;">Remove</a></li>
+<li id="attachment_add_2">
+ <a href="#" onclick="new Ajax.Request('/admin/content/attachment_box_add/2', {asynchronous:true, evalScripts:true, parameters:'authenticity_token=' + encodeURIComponent('nNxs7vtmzZz06VyUQI/fvfDba86FWg+fBlhZH3KB1KU=')}); return false;">Add Another Attachment</a></li>
+
+<li id="resources">
+
+ <h3>You can associate the following resources</h3>
+</li>
+ </ul>
+ </div>
+ </div>
+
+</div>
+
+
+
+</form>
+
+ </div>
+ </div>
+ </div>
+
+ <footer>
+ <a href="https://ancient-bastion-6361.herokuapp.com">c9-test-1</a> is proudly powered by Typo 6.1.0
+ </footer>
+</body>
+</html>
# ./submissions/assignment5-legacy-new-feature-spec/article_merge_feature_spec.rb:306:in `block (2 levels) in <top (required)>'
# ./lib/graders/rspec_grader/rspec_grader.rb:46:in `compute_points'
# ./lib/graders/rspec_grader/rspec_grader.rb:71:in `block in runner_block'
# ./lib/graders/rspec_grader/rspec_grader.rb:69:in `each'
# ./lib/graders/rspec_grader/rspec_grader.rb:69:in `runner_block'
# ./lib/auto_grader.rb:73:in `call'
# ./lib/auto_grader.rb:73:in `block in run_in_subprocess'
# ./lib/auto_grader.rb:69:in `fork'
# ./lib/auto_grader.rb:69:in `run_in_subprocess'
# ./lib/graders/rspec_grader/rspec_grader.rb:27:in `grade'
# ./lib/graders/rspec_grader/heroku_rspec_grader.rb:18:in `grade'
# ./lib/graders/rspec_grader/hw5_grader.rb:22:in `grade'
# ./lib/submission/base.rb:20:in `handle_submission'
# ./lib/submission/polling.rb:19:in `block in run'
# ./lib/submission/polling.rb:17:in `loop'
# ./lib/submission/polling.rb:17:in `run'
# run_autograder.rb:27:in `<main>'
Deprecation Warnings:
Using `should` from rspec-expectations' old `:should` syntax without explicitly enabling the syntax is deprecated. Use the new `:expect` syntax or explicitly enable `:should` with `config.expect_with(:rspec) { |c| c.syntax = :should }` instead. Called from /home/ubuntu/rag/submissions/assignment5-legacy-new-feature-spec/article_merge_feature_spec.rb:135:in `block (2 levels) in <top (required)>'.
If you need more of the backtrace for any of these deprecations to
identify where to make the necessary changes, you can configure
`config.raise_errors_for_deprecations!`, and it will turn the
deprecation warnings into errors, giving you the full backtrace.
Finished in 32.76 seconds (files took 2 minutes 50.2 seconds to load)
11 examples, 5 failures
Failed examples:
rspec ./submissions/assignment5-legacy-new-feature-spec/article_merge_feature_spec.rb:232 # The article merge feature should be shown on the article edit page [15 points]
rspec ./submissions/assignment5-legacy-new-feature-spec/article_merge_feature_spec.rb:242 # The article merge feature should create a single merged article [15 points]
rspec ./submissions/assignment5-legacy-new-feature-spec/article_merge_feature_spec.rb:262 # The article merge feature should create an article with the text of both original articles [20 points]
rspec ./submissions/assignment5-legacy-new-feature-spec/article_merge_feature_spec.rb:277 # The article merge feature should carry over the comments from both merged articles [20 points]
rspec ./submissions/assignment5-legacy-new-feature-spec/article_merge_feature_spec.rb:294 # The article merge feature should only show the merge button to administrators [15 points]
{"version":"3.3.2","examples":[{"description":"should respond to a simple request [0 points]","points":1,"status":"passed","pending_message":null},{"description":"should authenticate the supplied user as an administrator [0 points]","points":1,"status":"passed","pending_message":null},{"description":"should post new articles [0 points]","points":1,"status":"passed","pending_message":null},{"description":"should post comments on articles [0 points]","points":1,"status":"passed","pending_message":null},{"description":"create blog publisher users [0 points]","points":1,"status":"passed","pending_message":null},{"description":"should be shown on the article edit page [15 points]","points":1,"status":"failed","pending_message":null},{"description":"should create a single merged article [15 points]","points":1,"status":"failed","pending_message":null},{"description":"should create an article with the text of both original articles [20 points]","points":1,"status":"failed","pending_message":null},{"description":"should carry over the comments from both merged articles [20 points]","points":1,"status":"failed","pending_message":null},{"description":"should only show the merge button to administrators [15 points]","points":1,"status":"failed","pending_message":null},{"description":"should only allow administrators to merge articles [15 points]","points":1,"status":"passed","pending_message":null}],"summary":{"duration":32.764189637,"example_count":11,"failure_count":5,"pending_count":0},"summary_line":"11 examples, 5 failures"}Submission message after grade: Your submission was recorded at 2016-02-01 10:00:37 +0000 : submission is on time.
Score: 0/100
There was a fatal error with your submission. It either timed out or caused an exception.
It looks to me like the timeout is hitting before the process can finish and unfortunately this is cutting off output that would be useful for the students to see.
We could increase the grader timeout - but perhaps it would be better to ask students to do more manual testing up front? Of course if they had access to the actual specs and could run them locally then we could take all this pressure off the grader ....
from rag.
I do remember configuring different timeouts for different assignments in the past, but where is that setting now ...
from rag.
Found the grader timeout for the heroku grader set to 180 (I assume seconds), i.e. 3 minutes. I up'ed it to 360 to see what effect that would have. From the newly scrollable tmux log output I had what looked like the right sort of feedback for the student, without an immediate timeout message that I had seen before, so I was optimistic, but after a couple more minutes we got the same message back, i.e.
Deprecation Warnings:
There was a fatal error with your submission. It either timed out or caused an exception.
from rag.
Further investigation would be greatly aided by the ability to grade from the command line as mentioned here: #61
from rag.
Superficially it feels like the grader process is hanging after it has got the data back - not sure why the rspec output isn't sent straight back to the student. It is when things pass ... and it does on the Legacy HW1 - perhaps there is something about the way one of the Legacy HW2 checks works ...
from rag.
hmm, from the tmux output we have:
should be shown on the article edit page [15 points] (FAILED - 1)
should create a single merged article [15 points] (FAILED - 2)
should create an article with the text of both original articles [20 points] (FAILED - 3)
should carry over the comments from both merged articles [20 points] (FAILED - 4)
should only show the merge button to administrators [15 points] (FAILED - 5)
should only allow administrators to merge articles [15 points]
and then only output from the first 5 ... so maybe something about only allowing administrators to merge articles is getting stuck ...? or heroku is getting slow at that point ...
from rag.
I have changed my submission file to point to my C9 instance, instead of Heroku, so I can look at the logs. There is activity for 6 or 7 minutes. I have attached a file with the log in case it can be of any help.
from rag.
i compiled a similar profile of requests from my koding box showing only lines starting with "Starting" and "Completed"
https://gist.github.com/bill-auger/92c163d8f34abca04e5f
what it shows is that most of the requests go through as normal for about one minute then toward the end there is a pause for about two minutes and then two more requests come in - then after about two more minutes the grader reports the "timeout or exception" error to the coureware user
it also shows that several requests originate from several machines other than the grader
also i found a fully acceptable (100%) assignment from summer 2013 - the grader passed this one but with 2 failures - although the test suite appears to be identical
Failures:
1) Typo should post comments on articles [0 points]
Failure/Error: page = agent.get URI.join($host, 'comments?article_id=' + id)
Net::HTTP::Persistent::Error:
connection refused: localhost:3000
2) The article merge feature should carry over the comments from both merged articles [20 points]
Failure/Error: page = agent.get URI.join($host, 'comments?article_id=' + article_id)
Net::HTTP::Persistent::Error:
connection refused: localhost:3000
from rag.
Are we using cucumber_spinner
or launchy
?? Apparently the autograder works fine if the app pass all the tests. Maybe the problem happens when the autograder tries to display a page when there is an error. It's just an idea.
from rag.
thanks @jmorcate and @bill-auger - I think I have an angle of attack on this - I need to get the relevant issues into the https://github.com/saasbook/hw-refactoring-legacy-code-new-feature-ci repo ...
I've also been tracing the log to the my heroku instance as the grader works on it ... looks totally normal - and I can see the grader tmux output showing like the full 5 failures, sitting there, which should go straight back to the edX - but the grader sits there doing nothing - edX spinner spins, and I guess I'll get a time out error in a few minutes ... hmm
from rag.
yeah, fatal error - this is going to need some digging ...
from rag.
https://github.com/saasbook/hw-refactoring-legacy-code-new-feature-ci is dead link - did you mean
https://github.com/saasbook/hw-refactoring-legacy-code-new-feature ?
from rag.
no - there's a private repo and its build is failing - https://github.com/saasbook/hw-refactoring-legacy-code-ci/issues/2 - I'm going to give you access:
from rag.
i have added a similar requests profile for a fully completed assignment to same the gist as above
https://gist.github.com/bill-auger/92c163d8f34abca04e5f
it shows that all requests come in succession without any pause - curiously some of the requests from external machines do not occur - most notably the final two requests of the incomplete assignment which were preceded by a two minute pause and followed by another two minute pause before the grader responded to the user - i ran a reverse DNS on these external machines but no records were returned
from rag.
deployed fix to assignment spec which was to include the following:
RSpec::Matchers.define :include_no_diff do |expected|
match do |actual|
actual.include? expected
end
failure_message do |actual|
"expected page.body to include \"#{expected}\" but it did not"
end
failure_message_when_negated do |actual|
"expected page.body to not include \"#{expected}\" but it did"
end
end
and then use this throughout the spec instead of the direct include operator - this avoids the contents of the entire web page and diff being printed out and allows the grader to operate normally
from rag.
As requested by TA Mr-Jonze I've added a screenshot of the form in which the submit button (Check button) lost its functionality (won't submit the form anymore).
from rag.
for completeness ive asked a student of the course Dee-2015 to add the above interesting screenshot which shows the grader report rendering a Typo blog page inline
from rag.
Related Issues (20)
- tell learner whether failure was a timeout or an error
- tr count issue HOT 6
- Ruby Calisthenics support
- error on grader HOT 4
- running out of space on grader
- A locally correct answer that can't pass the auto-grader for HW4 HOT 10
- Locally passed test cases fail on auto-grader for HW5
- suppress output to stdout during tests
- add acceptance test for Sinatra Hangperson
- could mutation grader output indicate that mutations are taking place? HOT 1
- bugs on master HOT 4
- Github Grader
- unit acceptance test grader fails in strange way
- acceptance unit test assignment (feature grader) appears to give full marks for rspec when no rspec tests are present
- acceptance unit test grader has data being added in a migration
- Student feedback on controller coverage is different from local feedback
- Student gets no feedback on number of specs passed but others do HOT 29
- Legacy HW1 grader reported spinning HOT 6
- can timeout be made more robust?
- grader down (reported for HW4) HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from rag.