juliaci / coveragetools.jl Goto Github PK

This issue is used to trigger TagBot; feel free to unsubscribe.

If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.

If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!

We currently only track coverage information collated at the line level, but it might be useful to be able to build tools like ProfileView.jl that shows instead that information further broken down by the called types of the arguments. This would require support upstream in julia's codegen itself, as well as downstream, in the presentation tools

Hi all,

@traitfn functions (defined using SimpleTraits) are no longer counted as covered. (They used to be.)

Here's an example - it seems that even though the function body is being covered, the declaration isn't.

Any way we could restore the previous behavior? Thanks.

Under Julia 1.5 (I guess), some comment blocks within code, which are marked with """ .... """ , are counted as not covered. Julia 1.2, for example, provides much more accuracte evaluations, by correctly ignoring them. As an example, see the results for lyapunov.jl, which is part of the MatrixEquations.jl package.
The actual coverage for this package is almost 100%, while the reported one is 97%.

First of all thanks for this amazing package and all the functionality (and how easy it is to use it), it really helps getting code coverage.

For a while already I was wondering, if some certain cases might be covered as well, so I am not sure whether this is the right repo (I hope so).

We have a few macros for functions, so we have some cases with

@macro function f(x)
    x=x+2
   return x
end

See for example

https://codecov.io/gh/JuliaManifolds/ManifoldsBase.jl/src/master/src/EmbeddedManifold.jl#L190

There, the lines in the body of the function are covered, but the (macro+) signature are not. Can this maybe be fixed/added? That would be great, because for the mentioned package this is the only coverage missing.

On https://travis-ci.com/github/JuliaCI/CoverageTools.jl I now see the dreaded message

Builds have been temporarily disabled for public repositories due to a negative credit balance. Please go to the Plan page to replenish your credit balance or alter your Consume paid credits for OSS setting.

For another open source project, I successfully asked Travis for some additional credits, but it took > 1 week (and they only gave me part of what I asked for, with a promise that there might be more but that they need to "escalate" it to someone higher up... uhu).

Instead it may make more sense to switch this repo and others under JuliaCI to another CI provider. I'd suggest GitHub Actions.

I noticed that lines a la local a::Type are not marked as covered. For example line 795 in
https://app.codecov.io/gh/thofma/Hecke.jl/blob/ad036be62efe927d03345b7242e3d3e3a3c88cd0/src/QuadForm/Herm/Genus.jl
is marked as not covered, but obviously it is "called".

I think it would be nice if analyze_malloc() could (optionally) store the source code from the line being analysed in its MallocInfo result struct. I've just written a work-around that requires opening and searching through the source files, but analyze_malloc_files has basically already done this, since the source lines are in the .mem files. The option for additional lines of context would be nice too. Maybe something like a keyword argument context, default value 0 (current behaviour), 1 adds the source code line, 2 includes +/-1 line of context, etc.

Knowing you have a code coverage of 69% is helpful. However it doesn't help improve the code coverage and manually inspecting the coverage files is not very rewarding.

This would also make it easier to find false negatives in coverage.
For it to be useful, some type of lines (multi line comments ...) should not be counted as lines which can or can't be covered but a neutral third thing. If your coverage would change because you delete old comments that would be surprising.

Per this discussion, could someone with admin privileges please rename this repo to CoverageTools.jl?

cc: @vtjnash

Continue PR from JuliaCI/Coverage.jl#218
Partially implemented in #35
Moved from JuliaCI/Coverage.jl#44

Fixing JuliaCI/Coverage.jl#234 issue broke this packages tests https://github.com/JuliaCI/NanosoldierReports/blob/8edc023aaf03029c7887ee43b017a17c1bd7d4a5/pkgeval/by_date/2020-01/10/logs/CoverageTools/1.5.0-DEV-eb5410a703.log

There is something weird with the Travis setup on the JuliaCI organization. It’s using staging.travis-ci.com, which I think is the experimental version of Travis.

As a result, some things that work on travis-ci.org don’t work on staging.travis-ci.com. For example, submission to Coveralls works perfectly from travis-ci.org, but doesn’t work from staging.travis-ci.com.

Could we uninstall staging.travis-ci.com from JuliaCI and install the regular Travis (travis-ci.org) on all the JuliaCI repositories?

cc: @ararslan @StefanKarpinski

There are a lot of @info in the package. Although it is sometimes practical to watch, it would be great if it would be a controllable feature.
What do you think of passing a verbose kwarg? I could make a PR

https://github.com/mojaie/MolecularGraph.jl/pull/38/checks?check_run_id=1441476647

To make this more searchable, the stacktrace is:

ERROR: LoadError: Base.Meta.ParseError("invalid iteration specification")
Stacktrace:
 [1] parse(::String, ::Int64; greedy::Bool, raise::Bool, depwarn::Bool) at ./meta.jl:184
 [2] parse(::String, ::Int64) at ./meta.jl:176
 [3] amend_coverage_from_src!(::FileCoverage) at /home/runner/.julia/packages/CoverageTools/6Qc5H/src/CoverageTools.jl:189
 [4] process_file(::String, ::String) at /home/runner/.julia/packages/CoverageTools/6Qc5H/src/CoverageTools.jl:240
 [5] process_folder(::String) at /home/runner/.julia/packages/CoverageTools/6Qc5H/src/CoverageTools.jl:263
 [6] process_folder(::String) at /home/runner/.julia/packages/CoverageTools/6Qc5H/src/CoverageTools.jl:269
 [7] process_folder() at /home/runner/.julia/packages/CoverageTools/6Qc5H/src/CoverageTools.jl:255
 [8] top-level scope at /home/runner/work/_actions/julia-actions/julia-processcoverage/v1/main.jl:9
 [9] include(::Function, ::Module, ::String) at ./Base.jl:380
 [10] include(::Module, ::String) at ./Base.jl:368
 [11] exec_options(::Base.JLOptions) at ./client.jl:296
 [12] _start() at ./client.jl:506
in expression starting at /home/runner/work/_actions/julia-actions/julia-processcoverage/v1/main.jl:9

It's a known issue that function signatures might not be marked as covered, even if their body is covered. This is mainly attributed to the function inlining (as mentioned in the README file). This issue breaks the usefulness of the coverage numbers and always requires manual checking of all files.

One way to improve this (before there's a better solution via Julia itself) is to do a post-processing on the coverage files and mark the signature of all multi-line functions as covered if at least one line in their body is covered. This change will make the coverage information much more accurate and useful.

BoundsError: attempt to access 8-codeunit String at index [1:9]
Stacktrace:
  [1] checkbounds
    @ ./strings/basic.jl:216 [inlined]
  [2] getindex
    @ ./strings/string.jl:265 [inlined]
  [3] process_cov(filename::String, folder::String)
    @ CoverageTools ~/.julia/packages/CoverageTools/JSYpL/src/CoverageTools.jl:133
  [4] process_file(filename::String, folder::String)
    @ CoverageTools ~/.julia/packages/CoverageTools/JSYpL/src/CoverageTools.jl:232
  [5] process_folder(folder::String)
    @ CoverageTools ~/.julia/packages/CoverageTools/JSYpL/src/CoverageTools.jl:258
  [6] process_folder(folder::String) (repeats 2 times)
    @ CoverageTools ~/.julia/packages/CoverageTools/JSYpL/src/CoverageTools.jl:264
  [7] (::var"#3#4")()
    @ Main ~/test_retestitems.jl:39
  [8] with_logstate(f::Function, logstate::Any)
    @ Base.CoreLogging ./logging.jl:511
  [9] with_logger(f::Function, logger::Logging.ConsoleLogger)
    @ Base.CoreLogging ./logging.jl:623
 [10] top-level scope
    @ ~/script.jl:38

Where ~/script.jl:38 looks like:

    Logging.with_logger(Logging.ConsoleLogger(stderr, Logging.Warn)) do
        src_coverage = Coverage.process_folder("src")
        LCOV.writefile("lcov.info", src_coverage)
    end