ruby-numo / numo-linalg Goto Github PK

View Code? Open in Web Editor NEW

37.0 6.0 9.0 2.16 MB

Linear Algebra Library for Ruby/Numo::NArray

License: BSD 3-Clause "New" or "Revised" License

Ruby 79.64% HTML 0.01% C 9.18% C++ 11.15% Shell 0.01%

ruby numo linalg narray lapack blas matrix linear-algebra-library

numo-linalg's People

Contributors

Stargazers

Watchers

Forkers

metanest sonots mrkn yoshoku nicolasleger naitoh atul9 sztheory tpibaba

numo-linalg's Issues

Add expm operation

Hi numo authors
I'm currently working with kernels and a couple of them needs the expm operation. in my project, I tried to implemented this operation using the Padé approximation using Nmatrix library. The implementation doesn't work (the error rate was huge) and several months after, I change to your library. Your library doesn't support the expm operation so I solve this problem using the Pycall library to use the expm operation from scipy.linalg. The problem is the huge consumption of memory and the time used copying the matrix ruby -> python -> ruby with very large matrix. Could you add the expm operation to your library?
I can give you as little reference my old code:
https://github.com/ElenaRojano/NetAnalyzer/blob/master/lib/NetAnalyzer/numo_expansion.rb
In line 79 is defined the expm operation, currently invoking python to solve it. But the commented code has a few implementations (with old Nmatrix code, but I thisk that you can guess easily the similar operations) that I tried to use with the cited resources. I think that the best reference is the scipy itself:
https://github.com/scipy/scipy/blob/master/scipy/sparse/linalg/matfuncs.py
The function is in the line 550.

Also, I would like to cite you in my papers. Do you have any citation that I could use?
Thank you very much in advance
Pedro Seoane

"stack smashing detected" occur with liblapacke64

I used this test script.

$ cat test.rb
require "numo/linalg/autoloader"
require "datasets-numo-narray"
require "rumale"

p Numo::Linalg::Autoloader.libs

numeric_columns = [
  :bill_length_mm, :bill_depth_mm, :flipper_length_mm, :body_mass_g
]

penguins = Datasets::Penguins.new
array = penguins.filter_map do |record|
  values = numeric_columns.map do |column|
    record[column]
  end
  unless values.include?(nil)
    values
  end
end

x = Numo::NArray[*array]

x_std = Rumale::Preprocessing::StandardScaler.new.fit_transform(x)

pca = Rumale::Decomposition::PCA.new#(n_components: 4)
p pca.fit_transform(x_std)

First, I confirmed that the problem occurs with liblapacke64.

$ ruby test.rb 
["/usr/lib/x86_64-linux-gnu/libopenblas.so.0", "/usr/lib/x86_64-linux-gnu/liblapacke64.so.3.9.0"]
*** stack smashing detected ***: terminated
Aborted (core dumped)

Then, I tried the same test script with liblapacke.

$ sudo apt remove liblapacke64
...(snip)...
$ ruby test.rb
["/usr/lib/x86_64-linux-gnu/libopenblas.so.0", "/usr/lib/x86_64-linux-gnu/liblapacke.so.3.9.0"]
Numo::DFloat#shape=[342,2]
[[1.84075, 0.0476324], 
 [1.30485, -0.427722], 
 [1.36718, -0.15425], 
 [1.87608, -0.00204541], 
 [1.90895, 0.827996], 
 [1.76045, -0.350965], 
 [0.808889, 0.521724], 
 [1.83252, -0.768507], 
 [1.19246, 1.02478], 
 [1.72767, -0.787415], 
 [1.74049, -0.665515], 
 [1.78609, -0.235726], 
 [1.94543, 1.00977], 
 [1.56169, 0.596213], 
 [1.7371, -0.593942], 
 [1.56656, 0.0957962], 
 [0.794444, 1.30888], 
 [2.33955, -0.631749], 
 [0.992793, 1.98276], 
 [2.39412, -0.294074], 
 ...

The problem does not occur with liblapacke.

Cannot install/use on windows 10

Hi,

I am struggling to make numo-linalg (0.1.5) work in windows 10. I am using it successfully on macos. It is amazing BTW :-)

Notes:

I am using Msys2
I have installed numo-narray
I have installed openblas using pacman
I have also tried to compile it from source. Same results
I have installed lapack using pacman

libopenblas.a and liblapacke.a are both in C:\msys64\mingw64\lib so I install the gem with

gem install numo-linalg -- --with-openblas-dir=C:\msys64\mingw64\lib --with-lapack-lib=C:\msys64\mingw64\lib

The problem is that when I try default, level 1, 2 or 3, for example

require "numo/linalg/linalg"
Numo::Linalg::Loader.load_openblas "C:\\msys64\\mingw64\\lib"

I always get

RuntimeError: cannot find OpenBLAS library
from C:/Apps/rubies/Ruby22-x64/lib/ruby/gems/2.2.0/gems/numo-linalg-0.1.5/lib/numo/linalg/loader.rb:90:in `load_openblas'

Any idea?
Thanks

clblas/clblast

I wonder if clblas could be used instead of openblas for ruby-dnn

require "numo/linalg"
Numo::Linalg::Blas.dlopen("/usr/lib64/libclBLAS.so")
p Numo::Linalg::Loader.libs
...

i get

["libopenblas.so.0"]
Epoch 1/20000
/usr/local/lib64/ruby/gems/3.1.0/gems/numo-linalg-0.1.7/lib/numo/linalg/function.rb:26:in `sgemm': unknown symbol "cblas_sgemm" (RuntimeError)
        from /usr/local/lib64/ruby/gems/3.1.0/gems/numo-linalg-0.1.7/lib/numo/linalg/function.rb:26:in `call'
        from /usr/local/lib64/ruby/gems/3.1.0/gems/numo-linalg-0.1.7/lib/numo/linalg/function.rb:157:in `dot'
        from /usr/local/lib64/ruby/gems/3.1.0/gems/numo-narray-0.9.2.1/lib/numo/narray/extra.rb:1105:in `dot'
        from /usr/local/lib64/ruby/gems/3.1.0/gems/ruby-dnn-2.0.0/lib/dnn/core/functions/basic.rb:142:in `forward'
...

With clblast

require "numo/linalg"
Numo::Linalg::Blas.dlopen("/usr/lib64/libclblast.so")
p Numo::Linalg::Loader.libs

I get

["libopenblas.so.0"]
Epoch 1/20000
[========================================]  4/4 loss: -0.0000, accuracy: 0.7500
Epoch 2/20000
[========================================]  4/4 loss: -0.0000, accuracy: 0.2500
Epoch 3/20000
[========================================]  4/4 loss: -0.0000, accuracy: 0.2500
Epoch 4/20000
[========================================]  4/4 loss: -0.0000, accuracy: 0.0000
Epoch 5/20000
terminate called after throwing an instance of 'clblast::CLCudaAPIError'
  what():  OpenCL error: clCreateContext: -6
Aborted

In this case my GPU 2G memory usage goes from 0 to to about 75% increasingly, then drops to 0 after the error.

Build fails on Mac M1 / Ventura 13.3.1

My apologies for posting this. My knowledge of build flags is extremely out-of-date. I assume there are some environment variables missing when I run gem install numo-linalg. Any help is much appreciated.

I did try prefixing the command with MACOSX_DEPLOYMENT_TARGET=13.0 and MACOSX_DEPLOYMENT_TARGET=12.0

System information

$ uname -a
Darwin peterc-air 22.4.0 Darwin Kernel Version 22.4.0: Mon Mar  6 21:00:41 PST 2023; root:xnu-8796.101.5~3/RELEASE_ARM64_T8103 arm64
$ sw_vers 
ProductName:		macOS
ProductVersion:		13.3.1
BuildVersion:		22E261
$ clang --version
Apple clang version 14.0.3 (clang-1403.0.22.14.1)
Target: arm64-apple-darwin22.4.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

Warnings

During build there are 121 warnings nearly all like below. I'm not sure those matter:

warning: implicit conversion loses integer precision: 'size_t' (aka 'unsigned long') to 'blasint' (aka 'int') [-Wshorten-64-to-32]

Errors

linking shared-object numo/linalg/blas.bundle
Undefined symbols for architecture arm64:
  "_na_data_type", referenced from:
      _blas_s_sdot in blas_s.o
      _blas_s_snrm2 in blas_s.o
      _blas_s_sasum in blas_s.o
      _blas_s_dsdot in blas_s.o
      _blas_s_sdsdot in blas_s.o
      _blas_s_sswap in blas_s.o
      _blas_s_scopy in blas_s.o
      ...
  "_na_ndloop", referenced from:
      _blas_s_sdot in blas_s.o
      _blas_s_snrm2 in blas_s.o
      _blas_s_sasum in blas_s.o
      _blas_s_dsdot in blas_s.o
      _blas_s_sswap in blas_s.o
      _blas_s_scopy in blas_s.o
      _blas_s_ddot in blas_d.o
      ...
  "_na_ndloop3", referenced from:
      _blas_s_sdsdot in blas_s.o
      _blas_s_saxpy in blas_s.o
      _blas_s_srot in blas_s.o
      _blas_s_srotm in blas_s.o
      _blas_s_sscal in blas_s.o
      _blas_s_sgemv in blas_s.o
      _blas_s_strmv in blas_s.o
      ...
  "_nary_dup", referenced from:
      _blas_s_saxpy in blas_s.o
      _blas_s_srot in blas_s.o
      _blas_s_srotm in blas_s.o
      _blas_s_sscal in blas_s.o
      _blas_s_sgemv in blas_s.o
      _blas_s_strmv in blas_s.o
      _blas_s_ssymv in blas_s.o
      ...
  "_nary_eShapeError", referenced from:
      _blas_s_sdot in blas_s.o
      _blas_s_snrm2 in blas_s.o
      _blas_s_sasum in blas_s.o
      _blas_s_dsdot in blas_s.o
      _blas_s_sdsdot in blas_s.o
      _blas_s_sswap in blas_s.o
      _blas_s_scopy in blas_s.o
      ...
  "_nary_get_pointer_for_read", referenced from:
      _blas_s_srotm in blas_s.o
      _blas_s_drotm in blas_d.o
  "_numo_cDComplex", referenced from:
      _blas_s_zdotc in blas_z.o
      _blas_s_zdotu in blas_z.o
      _blas_s_dznrm2 in blas_z.o
      _blas_s_dzasum in blas_z.o
      _blas_s_zswap in blas_z.o
      _blas_s_zcopy in blas_z.o
      _blas_s_zaxpy in blas_z.o
      ...
  "_numo_cDFloat", referenced from:
      _blas_s_dsdot in blas_s.o
      _blas_s_ddot in blas_d.o
      _blas_s_dnrm2 in blas_d.o
      _blas_s_dasum in blas_d.o
      _blas_s_dswap in blas_d.o
      _blas_s_dcopy in blas_d.o
      _blas_s_daxpy in blas_d.o
      ...
  "_numo_cSComplex", referenced from:
      _blas_s_cdotc in blas_c.o
      _blas_s_cdotu in blas_c.o
      _blas_s_scnrm2 in blas_c.o
      _blas_s_scasum in blas_c.o
      _blas_s_cswap in blas_c.o
      _blas_s_ccopy in blas_c.o
      _blas_s_caxpy in blas_c.o
      ...
  "_numo_cSFloat", referenced from:
      _blas_s_sdot in blas_s.o
      _blas_s_snrm2 in blas_s.o
      _blas_s_sasum in blas_s.o
      _blas_s_dsdot in blas_s.o
      _blas_s_sdsdot in blas_s.o
      _blas_s_sswap in blas_s.o
      _blas_s_scopy in blas_s.o
      ...
  "_sym_init", referenced from:
      _blas_s_sgemv in blas_s.o
      _blas_s_strmv in blas_s.o
      _blas_s_ssymv in blas_s.o
      _blas_s_ssyr in blas_s.o
      _blas_s_sger in blas_s.o
      _blas_s_ssyr2 in blas_s.o
      _blas_s_sgemm in blas_s.o
      ...
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [blas.bundle] Error

run in rails occur crash, core dump

include rails Gemfile, start rails, it's crash and core dump

1497 /usr/local/lib/ruby/gems/2.5.0/gems/numo-linalg-0.1.4/lib/numo/linalg/lapack.so

Process memory map:

56331ff11000-563320235000 r-xp 00000000 fd:01 271503 /usr/local/bin/ruby
563320435000-56332043a000 r--p 00324000 fd:01 271503 /usr/local/bin/ruby
56332043a000-56332043b000 rw-p 00329000 fd:01 271503 /usr/local/bin/ruby
56332043b000-56332044d000 rw-p 00000000 00:00 0
5633209b0000-5633240af000 rw-p 00000000 00:00 0 [heap]
7f3e656d7000-7f3e6671f000 r--s 00000000 fd:01 271503 /usr/local/bin/ruby
7f3e6671f000-7f3e66758000 r-xp 00000000 fd:01 1838621 /usr/local/lib/ruby/gems/2.5.0/gems/numo-linalg-0.1.4/lib/numo/linalg/lapack.so
7f3e66758000-7f3e66957000 ---p 00039000 fd:01 1838621 /usr/local/lib/ruby/gems/2.5.0/gems/numo-linalg-0.1.4/lib/numo/linalg/lapack.so
7f3e66957000-7f3e66958000 r--p 00038000 fd:01 1838621 /usr/local/lib/ruby/gems/2.5.0/gems/numo-linalg-0.1.4/lib/numo/linalg/lapack.so
7f3e66958000-7f3e66959000 rw-p 00039000 fd:01 1838621 /usr/local/lib/ruby/gems/2.5.0/gems/numo-linalg-0.1.4/lib/numo/linalg/lapack.so
7f3e66959000-7f3e6695a000 rw-p 00000000 00:00 0
7f3e6695a000-7f3e6697c000 r-xp 00000000 fd:01 1838602 /usr/local/lib/ruby/gems/2.5.0/gems/numo-linalg-0.1.4/lib/numo/linalg/blas.so

Release a new version

Many functions have been updated and added. I would appreciate it if you could consider for releasing a new version.

Release the new version

@masa16 Could you please release the new version?

It's been 1.5 years since the latest version was released.

There are several changes that are waiting for the next release, especially making the build process improvement and the autoloader of the external libraries.

Crash when array is nil

require 'numo/narray'
require 'numo/linalg'
a = nil
b = Numo::DFloat.new(3,3).seq

Numo::Linalg.matmul(a,b)

[NOTE]
You may have encountered a bug in the Ruby interpreter or extension libraries.
Bug reports are welcome.
For details: http://www.ruby-lang.org/bugreport.html

中止 (コアダンプ)

numo-linalg doesn't build/install on Ubuntu 22.04 or Rocky Linux 9

Using gcc-11.2 on Ubuntu, gcc-8.5 on Rocky

Seems to have a problem with what looks like an undefined UNUSED() macro

compiling blas.c
ruby /root/.rbenv/versions/truffleruby+graalvm-22.3.0/graalvm/languages/ruby/lib/gems/gems/numo-linalg-0.1.7/ext/numo/linalg/blas/gen/cogen.rb -l -o blas_s.c s
compiling blas_s.c
/root/.rbenv/versions/truffleruby+graalvm-22.3.0/graalvm/languages/ruby/lib/gems/gems/numo-linalg-0.1.7/ext/numo/linalg/blas/gen/../tmpl/nrm2.c:36:57: error: expected ')'
blas_s_snrm2(int argc, VALUE const argv[], VALUE UNUSED(mod))
^
/root/.rbenv/versions/truffleruby+graalvm-22.3.0/graalvm/languages/ruby/lib/gems/gems/numo-linalg-0.1.7/ext/numo/linalg/blas/gen/../tmpl/nrm2.c:36:13: note: to match this '('
blas_s_snrm2(int argc, VALUE const argv[], VALUE UNUSED(mod))
^
/root/.rbenv/versions/truffleruby+graalvm-22.3.0/graalvm/languages/ruby/lib/gems/gems/numo-linalg-0.1.7/ext/numo/linalg/blas/gen/../tmpl/nrm2.c:34:57: error: expected ')'
blas_s_sasum(int argc, VALUE const argv[], VALUE UNUSED(mod))
^
/root/.rbenv/versions/truffleruby+graalvm-22.3.0/graalvm/languages/ruby/lib/gems/gems/numo-linalg-0.1.7/ext/numo/linalg/blas/gen/../tmpl/nrm2.c:34:13: note: to match this '('
blas_s_sasum(int argc, VALUE const argv[], VALUE UNUSED(mod))
^

inv of singular matrix returns some matrix without error

I tried this script.

require "numo/linalg"

a = Numo::DFloat[[0, 0],
                 [0, 0]]
p(Numo::Linalg.inv(a))

Result is here.

Numo::DFloat#shape=[2,2]
[[0, 0], 
 [0, 0]]

I think that a matrix with all zero elements has no inverse matrix.

Please release the new version

It's been 1.5 years since the latest version was released.

Installation on windows error

$ pkconfig --libs openblas
bash: pkconfig: command not found

Dominic E Sisneros@JZLCEWL01257106 MINGW64 ~
$ pkg-config.exe --libs openblas
-lopenblas

Dominic E Sisneros@JZLCEWL01257106 MINGW64 ~
$ pkg-config.exe --cflags openblas
-I/mingw64/include/OpenBLAS

Dominic E Sisneros@JZLCEWL01257106 MINGW64 ~
$ gem install numo-linalg
ERROR: Error installing numo-linalg:
ERROR: Failed to build gem native extension.

current directory: C:/Dominic/tools/msys64/mingw64/lib/ruby/gems/2.5.0/gems/                                                                                    numo-linalg-0.1.1/ext/numo/linalg/blas

C:/Dominic/tools/msys64/mingw64/bin/ruby.exe -r ./siteconf20180320-13684-5ljqfl. rb extconf.rb
checking for numo/narray.h... yes
checking for dlfcn.h... no
checking for windows.h... yes
checking for LoadLibrary()... yes
creating Makefile

current directory: C:/Dominic/tools/msys64/mingw64/lib/ruby/gems/2.5.0/gems/numo -linalg-0.1.1/ext/numo/linalg/blas
make "DESTDIR=" clean

current directory: C:/Dominic/tools/msys64/mingw64/lib/ruby/gems/2.5.0/gems/numo -linalg-0.1.1/ext/numo/linalg/blas
make "DESTDIR="
generating blas-x64-mingw32.def
compiling blas.c
blas.c: In function 'blas_s_dlopen':
blas.c:294:25: error: 'RTLD_LOCAL' undeclared (first use in this function); did you mean 'RTLD_GLOBAL'?
f = RTLD_LAZY | RTLD_LOCAL;
^~~~~~~~~~
RTLD_GLOBAL
blas.c:294:25: note: each undeclared identifier is reported only once for each f unction it appears in
make: *** [Makefile:249: blas.o] Error 1

$ ruby --version
ruby 2.5.0p0 (2017-12-25 revision 61468) [x64-mingw32]

RuntimeError: /usr/lib/libopenblas.so: undefined symbol: LAPACKE_dgesvd

When I use Ubuntu Xenial, Numo::Linalg.svd raises RuntimeError because it cannot resolve the symbol LAPACKE_dgesvd.

As I investigated libopenblas.so, I found that the library has dgesvd_ but not LAPACKE_dgesvd.

$ objdump -T /usr/lib/libopenblas.so | grep svd
0000000001aacb90 g    DF .text  0000000000008b48  Base        sgesvd_
000000000198ad60 g    DF .text  00000000000008e8  Base        dggsvd3_
0000000001ad0f50 g    DF .text  00000000000008e4  Base        sggsvd3_
000000000181fa30 g    DF .text  0000000000001e62  Base        cgesvdx_
0000000001939190 g    DF .text  0000000000001fe3  Base        dbdsvdx_
0000000001a7f960 g    DF .text  0000000000001f82  Base        sbdsvdx_
0000000001c11780 g    DF .text  00000000000007a5  Base        zggsvd_
0000000001beddd0 g    DF .text  00000000000088a4  Base        zgesvd_
000000000183a240 g    DF .text  0000000000000794  Base        cggsvd_
0000000001816f90 g    DF .text  0000000000008a94  Base        cgesvd_
0000000001bf6680 g    DF .text  0000000000001ee2  Base        zgesvdx_
000000000196f1d0 g    DF .text  0000000000001da5  Base        dgesvdx_
0000000001ab56e0 g    DF .text  0000000000001da5  Base        sgesvdx_
000000000198a5e0 g    DF .text  0000000000000775  Base        dggsvd_
000000000183a9e0 g    DF .text  0000000000000947  Base        cggsvd3_
00000000019667f0 g    DF .text  00000000000089dd  Base        dgesvd_
0000000001c11f30 g    DF .text  0000000000000953  Base        zggsvd3_
0000000001ad07d0 g    DF .text  0000000000000774  Base        sggsvd_