core.matrix.stats

This project is now DEPRECATED. Equivalent functionality has now been integrated into the main core.matrix library.

Overview

Statistical function library for Clojure using core.matrix

The purpose of this library is to provide useful statistical functions which are not part of core.matrix itself but are generally useful in statistical computing.

All of the functions are implemented on top of the core.matrix API: this means that you can safely use them with any core.matrix implementation, and you will gain the performance benefits of fast core.matrix implementations

core.matrix.stats is intended to be complementary with Incanter, and adopts the same conventions wherever possible.

Status

This project is now DEPRECATED. Equivalent functionality has now been integrated into the main core.matrix project

Usage

core.matrix.stats is available from Clojars:

https://clojars.org/net.mikera/core.matrix.stats

Ensure you have core.matrix.stats added to your project as a dependency, and it will be ready to use.

Examples

(use 'clojure.core.matrix.stats)

;; Calculate the mean of four vectors

(mean [[1 2 3] [4 5 6] [7 8 9] [10 11 12]])
=> [5.5 6.5 7.5]

;; calculate the sample variance of a set of numbers
(variance [1 2 3 4 5])
=> 2.5

Negative variance

Thanks for more good stuff. I love Clojure, but I often find myself wishing it was more like Matlab in some ways. The work you've done with core.matrix, et al., really pushes in that direction.

A couple issues, though. When computing variance across a sequence of 2-D matrices (actually, Clojure vectors of vectors) I'm occasionally getting negative values. Perhaps this is due to rounding issues? I've pasted in an example below. (I'm using [net.mikera/core.matrix.stats "0.3.0"])

Also, by way of a request/suggestion, it would be nice if the variance computation handled singletons gracefully. At the moment, passing a seq with a single value to the variance function returns NaN. It seems to me it should return 0. This would be nice, if there's no compelling reason not to return 0.

Here's the example data. The first form is the seq of 2 matrices I'm passing to core.matrix.stats/variance and core.matrix.stats/mean and core.matrix.stats/sd; the second form is a map of the results. I've also pasted, at the bottom, the code snippet which generates this map.

([[-0.073242 0.0583623 0.0493176 -0.0685103 0.0133153]
[-0.0238549 0.22721 0.00279091 -0.0295064 0.00345961]
[0.0474387 0.108434 -0.160608 0.0141515 -0.178786]
[-0.00671577 0.0193813 -0.171597 0.135594 -0.160267]
[-0.193098 -0.0406911 0.0117148 0.0172935 -0.09985]
[-0.0709003 0.0918066 0.226599 -0.0588987 -0.103077]
[-0.00691242 0.171046 0.163028 0.0684778 -0.0220451]
[-0.0117229 -0.0515936 0.0774852 0.0169544 -0.0130733]
[-0.00596545 -0.0584578 -0.0754406 0.0686553 -0.0336582]
[-0.0644147 -0.0608092 -0.356178 0.179706 -0.0663915]
[0.0510569 7.71253E-4 -0.418564 0.300115 -0.260289]
[-0.0543631 0.051265 -0.132305 0.065271 -0.334158]
[-0.0467448 -0.118863 0.0486948 0.19132 -0.0343882]
[-0.0676809 -0.217241 -0.0123118 0.0809029 0.120863]
[-0.115441 -0.0876327 -0.0861673 -0.0809323 0.274175]
[-0.0446653 0.146535 0.0838507 -0.137809 0.534201]
[0.0427561 0.249777 0.257593 -0.170756 0.408664]
[0.246956 -0.0440675 0.433791 -0.0463781 0.269146]
[0.435766 -0.190514 -0.0488814 0.122562 0.0180314]
[0.139139 -0.0578601 -0.281524 0.110487 -0.19655]]
[[-0.0114437 0.0386028 -0.035118 -0.0315776 0.014442]
[0.0106412 -0.0123212 -0.0826054 -0.0217214 0.0843651]
[0.0397683 -0.0909692 -0.0636879 0.00963786 0.216737]
[0.0761609 -0.128351 -0.0225831 -0.0279964 0.269716]
[0.0470947 -0.114555 0.0671166 -0.119323 0.1688]
[0.121403 -0.144795 0.123825 -0.0963807 -0.0461262]
[0.0519599 -0.184842 -0.0575316 0.133976 0.103778]
[0.0132536 -0.0710505 0.0673398 0.1983 0.437741]
[-0.0527778 -0.134378 0.376495 -0.137317 0.398581]
[-0.30744 -0.122783 0.30004 -0.306543 0.169034]
[-0.275664 -0.0782936 0.263279 -0.147994 0.464431]
[-0.112545 0.192529 0.43933 -0.0920934 0.632462]
[-0.0867552 0.474528 0.323681 -0.116266 0.0173113]
[-0.12675 0.169543 0.0534945 -0.120903 -0.585044]
[-0.0113475 -0.120233 0.0991003 0.00684753 -0.513591]
[0.0436613 -0.0590497 0.156853 0.0385004 -0.389928]
[-0.193536 -0.0706056 -0.123435 0.0545013 -0.589994]
[-0.17924 -0.112528 -0.510279 0.213063 -0.661463]
[0.243533 -0.190982 -0.427424 0.2597 -0.472558]
[0.129996 -0.182262 -0.106997 0.28745 -0.0127332]])

{:mean
#<NDArray [[-0.04234285 0.04848255 0.0070998 -0.05004395 0.01387865] [-0.00660
6849999999999 0.1074444 -0.039907245 -0.0256139 0.043912355] [0.0436035 0.008732
400000000001 -0.11214795 0.01189468 0.018975500000000006] [0.034722565000000004
-0.054484849999999994 -0.09709005 0.053798799999999994 0.05472450000000001] [-0.
07300165 -0.07762305 0.0394157 -0.05101475 0.034475000000000006] [0.02525135 -0.
026494200000000002 0.175212 -0.0776397 -0.0746016] [0.02252374 -0.00689800000000
0001 0.0527482 0.10122690000000001 0.04086645] [7.653500000000006E-4 -0.06132205
0.0724125 0.1076272 0.21233385] [-0.029371625 -0.0964179 0.15052720000000003 -0
.034330849999999996 0.1824614] [-0.18592735 -0.0917961 -0.02806900000000001 -0.0
634185 0.05132124999999999] [-0.11230355 -0.0387611735 -0.0776425 0.076060500000
00002 0.102071] [-0.08345405 0.121897 0.1535125 -0.013411200000000005 0.14915199
999999998] [-0.06675 0.1778325 0.1861879 0.037527 -0.00853845] [-0.0972154500000
0001 -0.023848999999999995 0.02059135 -0.02000005 -0.2320905] [-0.06339425 -0.10
393285 0.0064665 -0.037042385 -0.11970800000000001] [-5.01999999999999E-4 0.0437
4265 0.12035185 -0.0496543 0.07213650000000002] [-0.07538995000000001 0.08958569
999999999 0.067079 -0.058127349999999994 -0.090665] [0.033858 -0.07829775 -0.038
24400000000003 0.08334245 -0.1961585] [0.3396495 -0.190748 -0.23815270000000002
0.191131 -0.2272633] [0.1345675 -0.12006105 -0.1942605 0.1989685 -0.1046416]]>,
:count 2,
:variance
#<NDArray [[-0.07669687562255499 0.055151360858835 0.05045005960392 -0.0725219
49041445 0.013138637512355] [-0.023828965796404998 0.20427321378672 0.0064293857
02179949 -0.03034672452846 0.006720490254717951] [0.045217687260389994 0.1165568
8572912 -0.18170617677199502 0.013961421520774798 -0.1325312120315] [-0.00332660
03515484504 0.029918081441955 -0.18993995921239498 0.13058917665008 -0.093509821
14449999] [-0.201538571037355 -0.039618927757605 0.013112243182579999 0.02632646
8893874997 -0.07373361124999998] [-0.057436872944645 0.11136830675772001 0.18053
3140737 -0.06166530669969 -0.11208017111868002] [-0.005227226519165199 0.2051174
00156 0.16077313979208 0.06593359800878 -0.014615360187205002] [-0.0115484136082
85 -0.054066214082155006 0.07153270835154 0.03311006164032 0.08837255536935498]
[-0.00490533853744125 -0.058993175996819996 0.020991009145319997 0.0851540439655
55 0.05862428858108003] [-0.039033305356045 -0.06258658286142 -0.267729735922 0.
2656307985645 -0.04308674824712501] [0.101823366210795 0.0038962836587657966 -0.
3613048837715 0.3104468247155 -0.065429824321] [-0.055625879897805 0.05861465862
3 0.013573673587500001 0.07339247375267999 0.02135754323599997] [-0.048129460272
96 0.0430650266715 0.08413232154818003 0.20202123129799998 -0.034234329149115] [
-0.070517024937405 -0.18963372075299997 -0.010298145859395 0.094720431408995 0.3
554074815555] [-0.12334989610987501 -0.09478080032924499 -0.07643006178441 -0.08
362968790587555 0.509290704753] [-0.04275949489031 0.146195028212045 0.079484428
012155 -0.14125781821681999 0.6758374959194999] [0.06884499417399499 0.238710955
46238 0.26383001474300005 -0.174543185934355 0.740316635586] [0.276790249272 -0.
043666024526125 0.6912504507689999 -0.014874185975005 0.6297229861245] [0.264350
75638850003 -0.22680947468399998 0.02037645874142001 0.11694397167800002 0.13804
524831021997] [0.1198211359035 -0.05346997481020499 -0.3455499257115 0.113937574
51550002 -0.21828759451888]]>,
:sd
#<NDArray [[NaN 0.23484326871093197 0.22461090713480503 NaN 0.1146238959046280
1] [NaN 0.4519659431712969 0.08018345030104372 NaN 0.08197859632073454] [0.21264
450912353697 0.3414042848722318 NaN 0.11815845937035062 NaN] [NaN 0.172968440595
25715 NaN 0.3613712449131502 NaN] [NaN NaN 0.11450870352326935 0.162254333975629
13 NaN] [NaN 0.33371890380636215 0.42489191653525255 NaN NaN] [NaN 0.45289888513
44194 0.4009652600813193 0.25677538435134317 NaN] [NaN NaN 0.26745599329897246 0
.1819617037739535 0.2972752182227018] [NaN NaN 0.14488274274502122 0.29181165837
84051 0.24212453114271595] [NaN NaN NaN 0.5153938286053685 NaN] [0.3190977377086
7605 0.062420218349231976 NaN 0.5571775522358201 NaN] [NaN 0.24210464395174247 0
.11650610965739093 0.27091045338391795 0.14614220210466233] [NaN 0.2075211475283
9046 0.2900557214539648 0.44946771997330354 NaN] [NaN NaN NaN 0.3077668458573714
0.5961606172463089] [NaN NaN NaN NaN 0.7136460640632721] [NaN 0.382354584400455
44 0.2819298281703357 NaN 0.8220933620456377] [0.26238329629379037 0.48858055166
203657 0.5136438598318879 NaN 0.8604165477174414] [0.5261085907605008 NaN 0.8314
147285013659 NaN 0.7935508717936739] [0.5141505191950116 NaN 0.1427461338930761
0.34197071757388825 0.3715444096070078] [0.34615189715426953 NaN NaN 0.337546403
49957815 NaN]]>}

;; requiring core.matrix.stats in my ns declaration as follows:
(:require
[clojure.core.matrix.stats :as mstats])

(defn get-stats-map
[vals-seq](pprint/pprint vals-seq)
{:mean (mstats/mean vals-seq)
:count (count vals-seq)
:variance (mstats/variance vals-seq)
:sd (mstats/sd vals-seq)})

clojure-numerics / core.matrix.stats Goto Github PK