Comments (22)
can you give a bit more insight into it.
from carcv.
Our tests are being supervised by the cobertura-maven-plugin, then the coveralls-maven-plugin uses it to send the data to Coveralls.io.
The problem is, our integration tests (those running directly in the Java EE container) aren't being supervised by cobertura, therefore not seen in the coverage stats.
The aim is to make cobertura supervise integration tests too.
from carcv.
gotcha ill look into this.
from carcv.
We should continue all discussions here, so it's in one place. To continue on your line comment:
Yes, cobertura needs to be made aware of integration tests. I don't know how, haven't looked into it - not even sure if it's possible at all.
To give a bit more details, integration tests are mixed in the test directory with unit tests. You can spot an integration test by looking at the class name - it ends with "IT", as opposed to unit tests, which end in "Test". Most of the integration tests are in carcv-webapp, and are run with Arquillian on Wildfly.
As of my recent changes, carcv-core also has integration tests, although not run with Arquillian. Those changes also seem to have cause a problem with Travis I'm not able to fix yet, that's why the tests are "error"-ing. They run fine locally I think.
EDIT: I forgot to mention, integration tests are run through profiles - "it" in carcv-core, and for example "wildfly-it" in carcv-webapp (there are multiple there). If you want to run both, run mvn clean verify -Pit,wildfly-it
from the main directory.
They are also run in a completely different part of the maven lifecycle - in phase "integration-test" or "verify".
from carcv.
I've thought about this. Maybe it isn't such a good idea to include integration tests in the code coverage. That should probably just contain unit test coverage, as that's what it was designed to depict.
If we really need the test coverage for IT, we should probably switch to JaCoCo.
from carcv.
- That makes sense. I will post any updates on this thread.
- I looked into it a bit, i saw a few blog posts that said it should be possible, but have not come across a situation where i had to use integration tests to account for code coverage.I will post the helpful links out here on this.
- Makes sense, I was having a difficult time understanding the structure but now its a little more clearer.
- I am having a difficult time running the integration tests, have set up wildfly but for some reason it says cannot deploy, will post the stack trace shortly. Trying to understand the IT tests so that i can write a sample scala check code.
- May be true, when i looked at the link it seems pretty straightforward.
- Let me see if i can have any luck with cobertura, if i cannot then i will use JaCoCo.
from carcv.
You might want to take a look at this: https://github.com/oscarrenalias/scalacheck-examples/tree/master/scalacheck-integration-junit
from carcv.
when i try to run the integration tests with this command; i have set it up pwildfly according to integration tests set up.
i get the error attached.
The settings are being picked up by the parent project but for some reason the integration tests are failing travis build
working on it.
from carcv.
I've recently done some work on integration tests. You should pull the newest changes from develop as soon as you can.
If you're trying to exchange cobertura for JaCoCo, I'm not sure we need to. As I said above:
- I'm not sure about integration tests being counted into the coverage percentage
- JaCoCo can't handle branching coverage and line coverage counting AFAIK
Nevertheless, you should be able to run integration tests from the newest commit onward. If you get any errors after pulling, paste the whole log somewhere and we can work from there.
EDIT: I reran the build, everything seems to be fine. Passes locally too. If it doesn't work, tell me how you set up your environment and what OS / Java version you use.
from carcv.
JaCoCo seems to be able do it. Since we already have cobertura picking up the unit tests. I will try to see if it can pick up.
- You were right now the remote build is passing after rebase.
- But my local build still seems to fail. instructions i followed this to set up.
- but it again seems to give me the same problem as pasted above. I must be missing something very basic.
- i have java 7 and mavericks os on mac.
from carcv.
Ok. Can you paste the whole log somewhere? (From Maven)
from carcv.
log this is the log message.
- I just had a general question, how accurate is coveralls, because when i give the same commit from develop without any changes it still shows a coverage less than develop branch.
- Is there a time latency before coverage information can be trusted?
from carcv.
- The Coveralls question: I have no idea how they update the banner, but it seems to be pretty unreliable and not up to date. The latest coverage seems to be at around 32%. No idea how often and how they update.
- Regarding the log: I realized the docs were really misleading and to prevent future miss-directions like this, refactored and rewrote them today. The thing you are missing is a security-domain in wildfly's standalone.xml. Also, check the updated document on setting up the build environment, if you are missing something.
The hint in the error log was:
Caused by: java.lang.Exception: {"JBAS014771: Services with missing/unavailable dependencies" => ["jboss.undertow.deployment.default-server.default-host./carcv-webapp.UndertowDeploymentInfoService is missing [jboss.security.security-domain.carcv]"]}
Also, the cobertura checks fail. If you enabled them, it's quite expected -- they are set too strictly for our coverage right now.
from carcv.
standalone.xml; fixed the problem.
from carcv.
commit this should fix it. I am not sure why it fails when we have this phase enabled. At this time i am running out of ideas.
from carcv.
Are you sure we want to instrument in the process-classes
phase? I changed it to f.e. package
and the tests passed. I'm not sure I "fixed" it correctly - think you had a motivation for the process-classes
phase.
from carcv.
link this pom file has the same use case.
gist of the above linkusage.
- First we need to instrument the classes.
- Second we need to jar up the instrumented classes and have them used by the build later.
- Need to tell the Integration Tests to use the instrumented classes for itβs dependencies.
from carcv.
Sounds right. Last time I checked, you were just instrumenting them, right? Wouldn't that cause the tests to fail, because you aren't packaging them into the war file?
EDIT: Also, I'm not really comfortable with the thought of testing only the instrumented classes. We should test "normal" production code too.
from carcv.
ok. i wont follow that approach. I will keep this on hold and start with another feature; come back to this at a later point.
from carcv.
I agree. Keep the code & ideas, we might need that later on.
One solution to running instrumented and not instrumented classes in tests on their own would be to create separate profiles, but lets leave it as is for now.
from carcv.
I'm closing the issue as wontfix
for now in order to get back to it eventually.
Thank you for all the work and time you've put into this!
from carcv.
ok that sounds good.
from carcv.
Related Issues (20)
- Add `oraclejdk8` early access testing in Travis HOT 1
- Write tests for anything HOT 2
- Times should show the local timezone HOT 1
- CarCV fails to deploy on Wildfly8 on OpenShift HOT 1
- Release 1.0.2.Final HOT 1
- Implement functional testing? HOT 5
- Redesign the front-end HOT 2
- Design and draw a logo HOT 1
- Add, Edit, Delete toolbar for car table HOT 1
- Fill the info pages
- Add custom error handling pages
- Runtime error while recognizing
- Add support for integration testing inside docker for quicker setup
- Fix all integration test profiles (locally)
- Logging error on JBoss AS7 deploy
- Fix the RolesIT test HOT 2
- Fix deploy on Wildfly 8.2.0.Final
- numberplate-text=null in output file HOT 2
- DB script HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from carcv.