GithubHelp home page GithubHelp logo

Comments (21)

tansaku avatar tansaku commented on August 20, 2024

filesystem looks okay:

Filesystem      Size  Used Avail Use% Mounted on
/dev/xvda1      7.8G  2.9G  4.5G  39% /
proc               0     0     0    - /proc
sysfs              0     0     0    - /sys
none            4.0K     0  4.0K   0% /sys/fs/cgroup
none               0     0     0    - /sys/fs/fuse/connections
none               0     0     0    - /sys/kernel/debug
none               0     0     0    - /sys/kernel/security
udev            3.7G  8.0K  3.7G   1% /dev
devpts             0     0     0    - /dev/pts
tmpfs           752M  208K  752M   1% /run
none            5.0M     0  5.0M   0% /run/lock
none            3.7G     0  3.7G   0% /run/shm
none            100M     0  100M   0% /run/user
none               0     0     0    - /sys/fs/pstore
systemd            0     0     0    - /sys/fs/cgroup/systemd
/dev/xvdb        30G   45M   28G   1% /mnt

this from TA Mr. Jonze


mr-jonze [8:35 AM] 
this is the app i am submitting passes legacy-hw1 but gives fatal error for legacy-hw2 - you can use as a reference if you like --> aqueous-inlet-36299.herokuapp.com admin 0RHterL

[8:36] 
it seems intermittent has been reported several times over the past few days - some came back to the forum to say their went through tho

from rag.

tansaku avatar tansaku commented on August 20, 2024

mr-jonze has sent me his version of the software with some notes:


lemme know when you have it and ill destroy

[8:56] 
looking at the diff now i actually broke the edit.js path - not sure if that is relevant

[8:56] 
sry the new.js path

[8:58] 
and im not sure what that deal is with the files like "public/javascripts/ckeditor/config.bak" these got modified somehow so i blacklisted them

[9:00] 
fix the error i made on line 11 of categories controller if you think the js path maybe important https://github.com/bill-auger/legacy-hw1/commit/346d41dc37736e32e6d82bf0946bbdf6a758447e#diff-eb471b0b3c7c135b4709a71408d7bfabL11

[9:00] 
it scored 100% tho

in order to investigate further

from rag.

tansaku avatar tansaku commented on August 20, 2024

mr-jonze code grades fine on Legacy HW1, gets the following:

Your submission was recorded at 2016-02-01 09:47:56 +0000 : submission is on time.
Score: 0/100
There was a fatal error with your submission. It either timed out or caused an exception.

on Legacy HW2, while the grader log has the following:


  1) Typo should respond to a simple request [0 points]
     Failure/Error: page = agent.get($host)
     SocketError:
       getaddrinfo: Name or service not known

which I've seen on my own checks, but not leading to that timeout error ...

from rag.

tansaku avatar tansaku commented on August 20, 2024

I wonder if it is timeouts related to heroku apps sleeping ...

from rag.

tansaku avatar tansaku commented on August 20, 2024

I just got the same error from my shell app:

Your submission was recorded at 2016-02-01 10:00:37 +0000 : submission is on time.
Score: 0/100
There was a fatal error with your submission. It either timed out or caused an exception.

frustratingly the log output is minimal, while the output in the tmux window is more extensive, but scrolling is a real pain:


  5) The article merge feature should only show the merge button to administrators [15 points]
     Failure/Error: page.body.should include 'Merge Articles'
       expected "<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Transitional//EN\"\n  \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd\">\n<html xmlns=\"http://www.w3.org/19
99/xhtml\">\n<head>\n  <meta http-equiv=\"Content-type\" content=\"text/html; charset=utf-8\" />\n  <title>c9-test-1 \xE2\x80\x93 content</title>\n  <meta http-equiv=\"imagetoolbar\
" content=\"no\" />\n  <link href=\"/stylesheets/growler.css?1453218056\" media=\"screen\" rel=\"stylesheet\" type=\"text/css\" />\n<link href=\"/stylesheets/bootstrap.css?145321805
6\" media=\"screen\" rel=\"stylesheet\" type=\"text/css\" />\n<link href=\"/stylesheets/administration_structure.css?1453218056\" media=\"screen\" rel=\"stylesheet\" type=\"text/css
\" />\n<link href=\"/stylesheets/lightbox.css?1453218056\" media=\"screen\" rel=\"stylesheet\" type=\"text/css\" />\n  <script src=\"/javascripts/prototype.js?1453218056\" type=\"te
xt/javascript\"></script>\n<script src=\"/javascripts/effects.js?1453218056\" type=\"text/javascript\"></script>\n<script src=\"/javascripts/builder.js?1453218056\" type=\"text/java
script\"></script>\n<script src=\"/javascripts/dragdrop.js?1453218056\" type=\"text/javascript\"></script>\n<script src=\"/javascripts/controls.js?1453218056\" type=\"text/javascrip
t\"></script>\n<script src=\"/javascripts/typo.js?1453218056\" type=\"text/javascript\"></script>\n<script src=\"/javascripts/lightbox.js?1453218056\" type=\"text/javascript\"></scr
ipt>\n<script src=\"/javascripts/growler.js?1453218056\" type=\"text/javascript\"></script>\n<script src=\"/javascripts/typo_carousel.js?1453218056\" type=\"text/javascript\"></scri
pt>\n<script src=\"/javascripts/administration.js?1453218056\" type=\"text/javascript\"></script>\n<script src=\"/javascripts/ckeditor/ckeditor.js?1453218056\" type=\"text/javascrip
t\"></script>\n  <script src=\"/javascripts/calendar_date_select/calendar_date_select.js?1453218056\" type=\"text/javascript\"></script>\n<link href=\"/stylesheets/calendar_date_sel
ect/default.css?1453218056\" media=\"screen\" rel=\"stylesheet\" type=\"text/css\" />\n\n  <style type=\"text/css\">\n    #carousel-content .slide { width: 125; }\n  </style>\n</hea
d>\n<body>\n  <div class='topbar'>\n    <div class='topbar-inner'>\n      <div class='container-fluid'>\n        <a href=\"https://ancient-bastion-6361.herokuapp.com\" class=\"brand
\">c9-test-1</a>\n        <ul class=\"nav secondary-nav\">\n          <li class=\"dropdown\" onmouseover='$(\"user_menu\").style.display = \"block\"' onmouseout='$(\"user_menu\").st
yle.display = \"none\"'>\n            <a class=\"dropdown-toggle\" href=\"#\">Logged in as </a>\n            <ul class=\"dropdown-menu\" id=\"user_menu\" style='display: none'>\n   

...
       +      <span class='help-block'>Excerpts are posts summaries that are shown on your blog homepage only but won???t appear on the post itself</span>
       +    </div>
       +  </div>
       +
       +  <div class=''>
       +    <h4>Uploads</h4>
       +    <p class='help-block'>Uploads will be displayed as attachments in your RSS feed, but won't appear in your articles.</p>
       +    <ul id='attachments' class='inputs-list'>
       +      <li id="attachment_1">
       +  <input id="attachments_filename_1" name="attachments[filename_1]" type="file" />  <a href="#" onclick="if (confirm(_(&#x27;Really delete attachment&#x27;) +&#x27; 1?&#x27;)){
       +Effect.toggle(&quot;attachment_1&quot;,&#x27;appear&#x27;,{afterFinish:function(obj){Element.remove(obj.element);}});
       +}; return false;">Remove</a></li>
       +<li id="attachment_add_2">
       +  <a href="#" onclick="new Ajax.Request(&#x27;/admin/content/attachment_box_add/2&#x27;, {asynchronous:true, evalScripts:true, parameters:&#x27;authenticity_token=&#x27; + encodeURIComponent(&#x27;nNxs7vtmzZz06VyUQI/fvfDba86FWg+fBlhZH3KB1KU=&#x27;)}); return false;">Add Another Attachment</a></li>
       +
       +<li id="resources">
       +
       +  <h3>You can associate the following resources</h3>
       +</li>
       +    </ul>
       +  </div>    
       +  </div>
       +
       +</div>
       +
       +
       +
       +</form>
       +
       +      </div>
       +    </div>
       +  </div>
       +
       +  <footer>
       +    <a href="https://ancient-bastion-6361.herokuapp.com">c9-test-1</a> is proudly powered by Typo 6.1.0
       +  </footer>
       +</body>
       +</html>
     # ./submissions/assignment5-legacy-new-feature-spec/article_merge_feature_spec.rb:306:in `block (2 levels) in <top (required)>'
     # ./lib/graders/rspec_grader/rspec_grader.rb:46:in `compute_points'
     # ./lib/graders/rspec_grader/rspec_grader.rb:71:in `block in runner_block'
     # ./lib/graders/rspec_grader/rspec_grader.rb:69:in `each'
     # ./lib/graders/rspec_grader/rspec_grader.rb:69:in `runner_block'
     # ./lib/auto_grader.rb:73:in `call'
     # ./lib/auto_grader.rb:73:in `block in run_in_subprocess'
     # ./lib/auto_grader.rb:69:in `fork'
     # ./lib/auto_grader.rb:69:in `run_in_subprocess'
     # ./lib/graders/rspec_grader/rspec_grader.rb:27:in `grade'
     # ./lib/graders/rspec_grader/heroku_rspec_grader.rb:18:in `grade'
     # ./lib/graders/rspec_grader/hw5_grader.rb:22:in `grade'
     # ./lib/submission/base.rb:20:in `handle_submission'
     # ./lib/submission/polling.rb:19:in `block in run'
     # ./lib/submission/polling.rb:17:in `loop'
     # ./lib/submission/polling.rb:17:in `run'
     # run_autograder.rb:27:in `<main>'

Deprecation Warnings:

Using `should` from rspec-expectations' old `:should` syntax without explicitly enabling the syntax is deprecated. Use the new `:expect` syntax or explicitly enable `:should` with `config.expect_with(:rspec) { |c| c.syntax = :should }` instead. Called from /home/ubuntu/rag/submissions/assignment5-legacy-new-feature-spec/article_merge_feature_spec.rb:135:in `block (2 levels) in <top (required)>'.


If you need more of the backtrace for any of these deprecations to
identify where to make the necessary changes, you can configure
`config.raise_errors_for_deprecations!`, and it will turn the
deprecation warnings into errors, giving you the full backtrace.

Finished in 32.76 seconds (files took 2 minutes 50.2 seconds to load)
11 examples, 5 failures

Failed examples:

rspec ./submissions/assignment5-legacy-new-feature-spec/article_merge_feature_spec.rb:232 # The article merge feature should be shown on the article edit page [15 points]
rspec ./submissions/assignment5-legacy-new-feature-spec/article_merge_feature_spec.rb:242 # The article merge feature should create a single merged article [15 points]
rspec ./submissions/assignment5-legacy-new-feature-spec/article_merge_feature_spec.rb:262 # The article merge feature should create an article with the text of both original articles [20 points]
rspec ./submissions/assignment5-legacy-new-feature-spec/article_merge_feature_spec.rb:277 # The article merge feature should carry over the comments from both merged articles [20 points]
rspec ./submissions/assignment5-legacy-new-feature-spec/article_merge_feature_spec.rb:294 # The article merge feature should only show the merge button to administrators [15 points]
{"version":"3.3.2","examples":[{"description":"should respond to a simple request [0 points]","points":1,"status":"passed","pending_message":null},{"description":"should authenticate the supplied user as an administrator [0 points]","points":1,"status":"passed","pending_message":null},{"description":"should post new articles [0 points]","points":1,"status":"passed","pending_message":null},{"description":"should post comments on articles [0 points]","points":1,"status":"passed","pending_message":null},{"description":"create blog publisher users [0 points]","points":1,"status":"passed","pending_message":null},{"description":"should be shown on the article edit page [15 points]","points":1,"status":"failed","pending_message":null},{"description":"should create a single merged article [15 points]","points":1,"status":"failed","pending_message":null},{"description":"should create an article with the text of both original articles [20 points]","points":1,"status":"failed","pending_message":null},{"description":"should carry over the comments from both merged articles [20 points]","points":1,"status":"failed","pending_message":null},{"description":"should only show the merge button to administrators [15 points]","points":1,"status":"failed","pending_message":null},{"description":"should only allow administrators to merge articles [15 points]","points":1,"status":"passed","pending_message":null}],"summary":{"duration":32.764189637,"example_count":11,"failure_count":5,"pending_count":0},"summary_line":"11 examples, 5 failures"}Submission message after grade: Your submission was recorded at 2016-02-01 10:00:37 +0000 : submission is on time.
Score: 0/100
There was a fatal error with your submission. It either timed out or caused an exception.

It looks to me like the timeout is hitting before the process can finish and unfortunately this is cutting off output that would be useful for the students to see.

We could increase the grader timeout - but perhaps it would be better to ask students to do more manual testing up front? Of course if they had access to the actual specs and could run them locally then we could take all this pressure off the grader ....

from rag.

tansaku avatar tansaku commented on August 20, 2024

I do remember configuring different timeouts for different assignments in the past, but where is that setting now ...

from rag.

tansaku avatar tansaku commented on August 20, 2024

Found the grader timeout for the heroku grader set to 180 (I assume seconds), i.e. 3 minutes. I up'ed it to 360 to see what effect that would have. From the newly scrollable tmux log output I had what looked like the right sort of feedback for the student, without an immediate timeout message that I had seen before, so I was optimistic, but after a couple more minutes we got the same message back, i.e.

Deprecation Warnings:
There was a fatal error with your submission. It either timed out or caused an exception.

from rag.

tansaku avatar tansaku commented on August 20, 2024

Further investigation would be greatly aided by the ability to grade from the command line as mentioned here: #61

from rag.

tansaku avatar tansaku commented on August 20, 2024

Superficially it feels like the grader process is hanging after it has got the data back - not sure why the rspec output isn't sent straight back to the student. It is when things pass ... and it does on the Legacy HW1 - perhaps there is something about the way one of the Legacy HW2 checks works ...

from rag.

tansaku avatar tansaku commented on August 20, 2024

hmm, from the tmux output we have:

  should be shown on the article edit page [15 points] (FAILED - 1)
  should create a single merged article [15 points] (FAILED - 2)
  should create an article with the text of both original articles [20 points] (FAILED - 3)
  should carry over the comments from both merged articles [20 points] (FAILED - 4)
  should only show the merge button to administrators [15 points] (FAILED - 5)
  should only allow administrators to merge articles [15 points]

and then only output from the first 5 ... so maybe something about only allowing administrators to merge articles is getting stuck ...? or heroku is getting slow at that point ...

from rag.

jmorcate avatar jmorcate commented on August 20, 2024

I have changed my submission file to point to my C9 instance, instead of Heroku, so I can look at the logs. There is activity for 6 or 7 minutes. I have attached a file with the log in case it can be of any help.

c9_log_file.txt.zip

from rag.

bill-auger avatar bill-auger commented on August 20, 2024

i compiled a similar profile of requests from my koding box showing only lines starting with "Starting" and "Completed"

https://gist.github.com/bill-auger/92c163d8f34abca04e5f

what it shows is that most of the requests go through as normal for about one minute then toward the end there is a pause for about two minutes and then two more requests come in - then after about two more minutes the grader reports the "timeout or exception" error to the coureware user

it also shows that several requests originate from several machines other than the grader

also i found a fully acceptable (100%) assignment from summer 2013 - the grader passed this one but with 2 failures - although the test suite appears to be identical

Failures:

  1) Typo should post comments on articles [0 points]
     Failure/Error: page = agent.get URI.join($host, 'comments?article_id=' + id)
     Net::HTTP::Persistent::Error:
       connection refused: localhost:3000

  2) The article merge feature should carry over the comments from both merged articles [20 points]
     Failure/Error: page = agent.get URI.join($host, 'comments?article_id=' + article_id)
     Net::HTTP::Persistent::Error:
       connection refused: localhost:3000

from rag.

jmorcate avatar jmorcate commented on August 20, 2024

Are we using cucumber_spinner or launchy?? Apparently the autograder works fine if the app pass all the tests. Maybe the problem happens when the autograder tries to display a page when there is an error. It's just an idea.

from rag.

tansaku avatar tansaku commented on August 20, 2024

thanks @jmorcate and @bill-auger - I think I have an angle of attack on this - I need to get the relevant issues into the https://github.com/saasbook/hw-refactoring-legacy-code-new-feature-ci repo ...

I've also been tracing the log to the my heroku instance as the grader works on it ... looks totally normal - and I can see the grader tmux output showing like the full 5 failures, sitting there, which should go straight back to the edX - but the grader sits there doing nothing - edX spinner spins, and I guess I'll get a time out error in a few minutes ... hmm

from rag.

tansaku avatar tansaku commented on August 20, 2024

yeah, fatal error - this is going to need some digging ...

from rag.

bill-auger avatar bill-auger commented on August 20, 2024

https://github.com/saasbook/hw-refactoring-legacy-code-new-feature-ci is dead link - did you mean
https://github.com/saasbook/hw-refactoring-legacy-code-new-feature ?

from rag.

tansaku avatar tansaku commented on August 20, 2024

no - there's a private repo and its build is failing - https://github.com/saasbook/hw-refactoring-legacy-code-ci/issues/2 - I'm going to give you access:

from rag.

bill-auger avatar bill-auger commented on August 20, 2024

i have added a similar requests profile for a fully completed assignment to same the gist as above

https://gist.github.com/bill-auger/92c163d8f34abca04e5f

it shows that all requests come in succession without any pause - curiously some of the requests from external machines do not occur - most notably the final two requests of the incomplete assignment which were preceded by a two minute pause and followed by another two minute pause before the grader responded to the user - i ran a reverse DNS on these external machines but no records were returned

from rag.

tansaku avatar tansaku commented on August 20, 2024

deployed fix to assignment spec which was to include the following:

RSpec::Matchers.define :include_no_diff do |expected|
  match do |actual|
    actual.include? expected
  end

  failure_message do |actual|
    "expected page.body to include \"#{expected}\" but it did not"
  end

  failure_message_when_negated do |actual|
    "expected page.body to not include \"#{expected}\" but it did"
  end
end

and then use this throughout the spec instead of the direct include operator - this avoids the contents of the entire web page and diff being printed out and allows the grader to operate normally

from rag.

Dee-2015 avatar Dee-2015 commented on August 20, 2024

As requested by TA Mr-Jonze I've added a screenshot of the form in which the submit button (Check button) lost its functionality (won't submit the form anymore).
courses edx org_2016-02-03_09-24-02

from rag.

bill-auger avatar bill-auger commented on August 20, 2024

for completeness ive asked a student of the course Dee-2015 to add the above interesting screenshot which shows the grader report rendering a Typo blog page inline

from rag.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.