Comments (2)
jwijffels
commented on May 26, 2024
==2295052== Memcheck, a memory error detector
==2295052== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al.
==2295052== Using Valgrind-3.21.0 and LibVEX; rerun with -h for copyright info
==2295052== Command: /data/blackswan/ripley/R/R-devel-vg/bin/exec/R --vanilla
==2295052==
R Under development (unstable) (2023-08-14 r84947) -- "Unsuffered Consequences"
Copyright (C) 2023 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu
R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.
Natural language support but running in an English locale
R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.
Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.
> pkgname <- "crfsuite"
> source(file.path(R.home("share"), "R", "examples-header.R"))
> options(warn = 1)
> library('crfsuite')
>
> base::assign(".oldSearch", base::search(), pos = 'CheckExEnv')
> base::assign(".old_wd", base::getwd(), pos = 'CheckExEnv')
> cleanEx()
> nameEx("airbnb")
> ### * airbnb
>
> flush(stderr()); flush(stdout())
>
> ### Name: airbnb
> ### Title: Dutch reviews of AirBnB customers on Brussels address locations
> ### available at www.insideairbnb.com
> ### Aliases: airbnb
>
> ### ** Examples
>
> data(airbnb)
> str(airbnb)
'data.frame': 500 obs. of 3 variables:
$ doc_id : int 26261897 11412934 19782360 23591517 11462456 17644323 34988356 27922910 16041980 24277878 ...
$ listing_id: int 1405333 737088 184201 2015592 424311 2527721 3659296 4414049 2252840 1396647 ...
$ text : chr "Fijn, ruim appartement. Locatie vlakbij Manneken Pis en de Grote Markt, dus uitstekend gelegen.\nBadkamer was s"| __truncated__ "Het appartement van Salvatore ligt midden in de trendy modewijk. Het is een ideale locatie om Brussel te verken"| __truncated__ "Een prachtige kamer! Heel fijn was dat de gordijnen heel donker zijn waardoor je goed kunt slapen. David was he"| __truncated__ "Heel mooi huisje in centrum Brussel. Op wandelafstand van place St Catherine en dus vele leuke restaurantjes en kroegjes. " ...
> head(airbnb)
doc_id listing_id
1388 26261897 1405333
1115 11412934 737088
1112 19782360 184201
1261 23591517 2015592
1342 11462456 424311
1172 17644323 2527721
text
1388 Fijn, ruim appartement. Locatie vlakbij Manneken Pis en de Grote Markt, dus uitstekend gelegen.\nBadkamer was schoon maar had wel een muffe geur van vocht. Aline beloofde er naar te kijken.
1115 Het appartement van Salvatore ligt midden in de trendy modewijk. Het is een ideale locatie om Brussel te verkennen. Eigenlijk alles op loopafstand en als je dat liever doet de metro vlakbij.\nSalvatore is vriendelijk en behulpzaam, en het appartement van alle gemakken voorzien. Heerlijk rustig geslapen in het comfortabele bed.\nAls je met de auto komt, maak dan vooral gebruik van het parkeerabonnement van Salvatore in de parking Lepage!
1112 Een prachtige kamer! Heel fijn was dat de gordijnen heel donker zijn waardoor je goed kunt slapen. David was heel vriendelijk, al heb ik hem nauwelijks gesproken. Enig minpuntje was dat door vertraging met de trein van David ik op de afgesproken tijd 's avonds voor een dichte deur stond en David juist op dat moment geen bereik had met zijn telefoon waardoor het leek alsof die uit stond. Toen een vriend van hem uiteindelijk kwam opendoen, moest hij eerst nog de kamer in orde maken. Maar verder alles prima. Zeker voor een vakantie in Brussel een echte aanrader.
1261 Heel mooi huisje in centrum Brussel. Op wandelafstand van place St Catherine en dus vele leuke restaurantjes en kroegjes.
1342 Ideale ligging vlakbij de grote markt. Perfect om de de stad in te trekken voor cultuur, winkelen of nachtleven... \nWij hadden slechts een verblijf van 1 nacht tussen professionele afspraken door. De verhuurder bood ons grote flexibiliteit inzake komen en gaan. De boeking verliep trouwens super snel en werd heel goed opgevolgd. \nHet appartement zelf biedt alle nodige comfort voor een verblijf in de stad. Zeer netjes en alles was aanwezig qua handdoeken, beddengoed, keukengerei,... \nSlecht gelegen voor mensen met weinig discipline inzake avondlijke snacks: schitterende frituren binnen een straal van 50m ;-)
1172 Het apartement is licht, ruim, schoon, prima op orde en voorzien van alle benodigdheden. Het centrum, de metro en supermarkt zijn op loopafstand. \nDit was een prima uitvalsbasis voor ons bezoek aan Brussels en een fijne plek om naderhand te relaxen en eten.\nIn-, uitchecken en contact met de verhuurder verliep prima.\nWij zijn meer dan tevreden!
>
>
>
> cleanEx()
> nameEx("airbnb_chunks")
> ### * airbnb_chunks
>
> flush(stderr()); flush(stdout())
>
> ### Name: airbnb_chunks
> ### Title: Dutch reviews of AirBnB customers on Brussels address locations
> ### manually tagged with entities
> ### Aliases: airbnb_chunks
>
> ### ** Examples
>
> data(airbnb_chunks)
> str(airbnb_chunks)
Classes ‘chunkrange’ and 'data.frame': 1091 obs. of 8 variables:
$ annotation_time: POSIXct, format: "2018-09-02 21:47:53" "2018-09-02 21:47:59" ...
$ doc_id : int 26261897 26261897 26261897 11412934 11412934 11412934 11412934 19782360 19782360 19782360 ...
$ text : chr "Fijn, ruim appartement. Locatie vlakbij Manneken Pis en de Grote Markt, dus uitstekend gelegen.\nBadkamer was s"| __truncated__ "Fijn, ruim appartement. Locatie vlakbij Manneken Pis en de Grote Markt, dus uitstekend gelegen.\nBadkamer was s"| __truncated__ "Fijn, ruim appartement. Locatie vlakbij Manneken Pis en de Grote Markt, dus uitstekend gelegen.\nBadkamer was s"| __truncated__ "Het appartement van Salvatore ligt midden in de trendy modewijk. Het is een ideale locatie om Brussel te verken"| __truncated__ ...
$ start : num 41 59 156 21 95 192 424 100 220 292 ...
$ end : num 52 70 160 29 101 200 437 104 224 296 ...
$ chunk_id : int 1 2 3 4 5 6 7 8 9 10 ...
$ chunk_entity : chr "LOCATION" "LOCATION" "PERSON" "PERSON" ...
$ chunk : chr "Manneken Pis" " Grote Markt" "Aline" "Salvatore" ...
> head(airbnb_chunks)
annotation_time doc_id
1 2018-09-02 21:47:53 26261897
2 2018-09-02 21:47:59 26261897
3 2018-09-02 21:48:07 26261897
4 2018-09-02 21:48:15 11412934
5 2018-09-02 21:48:41 11412934
6 2018-09-02 21:48:48 11412934
text
1 Fijn, ruim appartement. Locatie vlakbij Manneken Pis en de Grote Markt, dus uitstekend gelegen.\nBadkamer was schoon maar had wel een muffe geur van vocht. Aline beloofde er naar te kijken.
2 Fijn, ruim appartement. Locatie vlakbij Manneken Pis en de Grote Markt, dus uitstekend gelegen.\nBadkamer was schoon maar had wel een muffe geur van vocht. Aline beloofde er naar te kijken.
3 Fijn, ruim appartement. Locatie vlakbij Manneken Pis en de Grote Markt, dus uitstekend gelegen.\nBadkamer was schoon maar had wel een muffe geur van vocht. Aline beloofde er naar te kijken.
4 Het appartement van Salvatore ligt midden in de trendy modewijk. Het is een ideale locatie om Brussel te verkennen. Eigenlijk alles op loopafstand en als je dat liever doet de metro vlakbij.\nSalvatore is vriendelijk en behulpzaam, en het appartement van alle gemakken voorzien. Heerlijk rustig geslapen in het comfortabele bed.\nAls je met de auto komt, maak dan vooral gebruik van het parkeerabonnement van Salvatore in de parking Lepage!
5 Het appartement van Salvatore ligt midden in de trendy modewijk. Het is een ideale locatie om Brussel te verkennen. Eigenlijk alles op loopafstand en als je dat liever doet de metro vlakbij.\nSalvatore is vriendelijk en behulpzaam, en het appartement van alle gemakken voorzien. Heerlijk rustig geslapen in het comfortabele bed.\nAls je met de auto komt, maak dan vooral gebruik van het parkeerabonnement van Salvatore in de parking Lepage!
6 Het appartement van Salvatore ligt midden in de trendy modewijk. Het is een ideale locatie om Brussel te verkennen. Eigenlijk alles op loopafstand en als je dat liever doet de metro vlakbij.\nSalvatore is vriendelijk en behulpzaam, en het appartement van alle gemakken voorzien. Heerlijk rustig geslapen in het comfortabele bed.\nAls je met de auto komt, maak dan vooral gebruik van het parkeerabonnement van Salvatore in de parking Lepage!
start end chunk_id chunk_entity chunk
1 41 52 1 LOCATION Manneken Pis
2 59 70 2 LOCATION Grote Markt
3 156 160 3 PERSON Aline
4 21 29 4 PERSON Salvatore
5 95 101 5 LOCATION Brussel
6 192 200 6 PERSON Salvatore
>
>
>
> cleanEx()
> nameEx("crf")
> ### * crf
>
> flush(stderr()); flush(stdout())
>
> ### Name: crf
> ### Title: Linear-chain Conditional Random Field
> ### Aliases: crf
>
> ### ** Examples
>
> ## Download modeldata (conll 2002 shared task in Dutch)
> # for CRAN only - word on a subset of the data
> x <- ner_download_modeldata("conll2002-nl", docs = 10)
trying URL 'https://raw.githubusercontent.com/bnosac-dev/nltk_data/gh-pages/packages/corpora/conll2002.zip'
Content type 'application/zip' length 1867449 bytes (1.8 MB)
==================================================
downloaded 1.8 MB
> if(is.data.frame(x)){
+ ##
+ ## Build Named Entity Recognition model on conll2002-nl
+ ##
+ x$pos <- txt_sprintf("Parts of Speech: %s", x$pos)
+ x$token <- txt_sprintf("Token: %s", x$token)
+ crf_train <- subset(x, data == "ned.train")
+ crf_test <- subset(x, data == "testa")
+
+ model <- crf(y = crf_train$label,
+ x = crf_train[, c("token", "pos")],
+ group = crf_train$doc_id,
+ method = "lbfgs",
+ options = list(max_iterations = 3, feature.minfreq = 5,
+ c1 = 0, c2 = 1))
+ model
+ weights <- coefficients(model)
+ head(weights$states, n = 20)
+ head(weights$transitions, n = 20)
+ stats <- summary(model, "modeldetails.txt")
+ stats
+ plot(stats$iterations$loss)
+
+ ## Use the CRF model to label a sequence
+ scores <- predict(model,
+ newdata = crf_test[, c("token", "pos")],
+ group = crf_test$doc_id)
+ head(scores)
+ crf_test$label <- scores$label
+
+ ## cleanup for CRAN
+ if(file.exists(model$file_model)) file.remove(model$file_model)
+ if(file.exists("modeldetails.txt")) file.remove("modeldetails.txt")
+ }
==2295052== realloc() with size 0
==2295052== at 0x4846A40: realloc (/builddir/build/BUILD/valgrind-3.21.0/coregrind/m_replacemalloc/vg_replace_malloc.c:1649)
==2295052== by 0x18220F41: mem_mgr (packages/tests-vg/crfsuite/src/crf/src/rumavl.c:1107)
==2295052== by 0x18220F41: stack_update (packages/tests-vg/crfsuite/src/crf/src/rumavl.c:922)
==2295052== by 0x182214AE: rumavl_set (packages/tests-vg/crfsuite/src/crf/src/rumavl.c:374)
==2295052== by 0x1822154C: rumavl_insert (packages/tests-vg/crfsuite/src/crf/src/rumavl.c:392)
==2295052== by 0x18220A31: quark_get (packages/tests-vg/crfsuite/src/crf/src/quark.c:117)
==2295052== by 0x1822650C: CRFSuite::Trainer::append(std::vector<std::vector<CRFSuite::Attribute, std::allocator<CRFSuite::Attribute> >, std::allocator<std::vector<CRFSuite::Attribute, std::allocator<CRFSuite::Attribute> > > > const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, int) (packages/tests-vg/crfsuite/src/./include/crfsuite.hpp:133)
==2295052== by 0x18229756: crfsuite_model_build(char const*, std::vector<int, std::allocator<int> >, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, Rcpp::Matrix<16, Rcpp::PreserveStorage>, Rcpp::Matrix<14, Rcpp::PreserveStorage>, Rcpp::Vector<19, Rcpp::PreserveStorage>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, int) (packages/tests-vg/crfsuite/src/rcpp_crfsuite.cpp:103)
==2295052== by 0x18236962: _crfsuite_crfsuite_model_build (packages/tests-vg/crfsuite/src/RcppExports.cpp:50)
==2295052== by 0x4A3BFD: R_doDotCall (svn/R-devel/src/main/dotcode.c:904)
==2295052== by 0x4A4203: do_dotcall (svn/R-devel/src/main/dotcode.c:1551)
==2295052== by 0x4DD026: bcEval (svn/R-devel/src/main/eval.c:7567)
==2295052== by 0x4F595F: Rf_eval (svn/R-devel/src/main/eval.c:1146)
==2295052== Address 0x1cb1a320 is 0 bytes inside a block of size 24 alloc'd
==2295052== at 0x484182F: malloc (/builddir/build/BUILD/valgrind-3.21.0/coregrind/m_replacemalloc/vg_replace_malloc.c:431)
==2295052== by 0x18221419: mem_mgr (packages/tests-vg/crfsuite/src/crf/src/rumavl.c:1107)
==2295052== by 0x18221419: stack_push (packages/tests-vg/crfsuite/src/crf/src/rumavl.c:847)
==2295052== by 0x18221419: rumavl_set (packages/tests-vg/crfsuite/src/crf/src/rumavl.c:342)
==2295052== by 0x1822154C: rumavl_insert (packages/tests-vg/crfsuite/src/crf/src/rumavl.c:392)
==2295052== by 0x18220A31: quark_get (packages/tests-vg/crfsuite/src/crf/src/quark.c:117)
==2295052== by 0x1822650C: CRFSuite::Trainer::append(std::vector<std::vector<CRFSuite::Attribute, std::allocator<CRFSuite::Attribute> >, std::allocator<std::vector<CRFSuite::Attribute, std::allocator<CRFSuite::Attribute> > > > const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, int) (packages/tests-vg/crfsuite/src/./include/crfsuite.hpp:133)
==2295052== by 0x18229756: crfsuite_model_build(char const*, std::vector<int, std::allocator<int> >, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, Rcpp::Matrix<16, Rcpp::PreserveStorage>, Rcpp::Matrix<14, Rcpp::PreserveStorage>, Rcpp::Vector<19, Rcpp::PreserveStorage>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, int) (packages/tests-vg/crfsuite/src/rcpp_crfsuite.cpp:103)
==2295052== by 0x18236962: _crfsuite_crfsuite_model_build (packages/tests-vg/crfsuite/src/RcppExports.cpp:50)
==2295052== by 0x4A3BFD: R_doDotCall (svn/R-devel/src/main/dotcode.c:904)
==2295052== by 0x4A4203: do_dotcall (svn/R-devel/src/main/dotcode.c:1551)
==2295052== by 0x4DD026: bcEval (svn/R-devel/src/main/eval.c:7567)
==2295052== by 0x4F595F: Rf_eval (svn/R-devel/src/main/eval.c:1146)
==2295052== by 0x4F7751: R_execClosure (svn/R-devel/src/main/eval.c:2320)
==2295052==
==2295052== realloc() with size 0
==2295052== at 0x4846A40: realloc (/builddir/build/BUILD/valgrind-3.21.0/coregrind/m_replacemalloc/vg_replace_malloc.c:1649)
==2295052== by 0x18220E6E: mem_mgr (packages/tests-vg/crfsuite/src/crf/src/rumavl.c:1107)
==2295052== by 0x18220E6E: stack_update (packages/tests-vg/crfsuite/src/crf/src/rumavl.c:930)
==2295052== by 0x182214AE: rumavl_set (packages/tests-vg/crfsuite/src/crf/src/rumavl.c:374)
==2295052== by 0x1822154C: rumavl_insert (packages/tests-vg/crfsuite/src/crf/src/rumavl.c:392)
==2295052== by 0x18220A31: quark_get (packages/tests-vg/crfsuite/src/crf/src/quark.c:117)
==2295052== by 0x1822650C: CRFSuite::Trainer::append(std::vector<std::vector<CRFSuite::Attribute, std::allocator<CRFSuite::Attribute> >, std::allocator<std::vector<CRFSuite::Attribute, std::allocator<CRFSuite::Attribute> > > > const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, int) (packages/tests-vg/crfsuite/src/./include/crfsuite.hpp:133)
==2295052== by 0x18229756: crfsuite_model_build(char const*, std::vector<int, std::allocator<int> >, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, Rcpp::Matrix<16, Rcpp::PreserveStorage>, Rcpp::Matrix<14, Rcpp::PreserveStorage>, Rcpp::Vector<19, Rcpp::PreserveStorage>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, int) (packages/tests-vg/crfsuite/src/rcpp_crfsuite.cpp:103)
==2295052== by 0x18236962: _crfsuite_crfsuite_model_build (packages/tests-vg/crfsuite/src/RcppExports.cpp:50)
==2295052== by 0x4A3BFD: R_doDotCall (svn/R-devel/src/main/dotcode.c:904)
==2295052== by 0x4A4203: do_dotcall (svn/R-devel/src/main/dotcode.c:1551)
==2295052== by 0x4DD026: bcEval (svn/R-devel/src/main/eval.c:7567)
==2295052== by 0x4F595F: Rf_eval (svn/R-devel/src/main/eval.c:1146)
==2295052== Address 0x1cb1ac10 is 0 bytes inside a block of size 24 alloc'd
==2295052== at 0x484182F: malloc (/builddir/build/BUILD/valgrind-3.21.0/coregrind/m_replacemalloc/vg_replace_malloc.c:431)
==2295052== by 0x18221419: mem_mgr (packages/tests-vg/crfsuite/src/crf/src/rumavl.c:1107)
==2295052== by 0x18221419: stack_push (packages/tests-vg/crfsuite/src/crf/src/rumavl.c:847)
==2295052== by 0x18221419: rumavl_set (packages/tests-vg/crfsuite/src/crf/src/rumavl.c:342)
==2295052== by 0x1822154C: rumavl_insert (packages/tests-vg/crfsuite/src/crf/src/rumavl.c:392)
==2295052== by 0x18220A31: quark_get (packages/tests-vg/crfsuite/src/crf/src/quark.c:117)
==2295052== by 0x1822650C: CRFSuite::Trainer::append(std::vector<std::vector<CRFSuite::Attribute, std::allocator<CRFSuite::Attribute> >, std::allocator<std::vector<CRFSuite::Attribute, std::allocator<CRFSuite::Attribute> > > > const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, int) (packages/tests-vg/crfsuite/src/./include/crfsuite.hpp:133)
==2295052== by 0x18229756: crfsuite_model_build(char const*, std::vector<int, std::allocator<int> >, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, Rcpp::Matrix<16, Rcpp::PreserveStorage>, Rcpp::Matrix<14, Rcpp::PreserveStorage>, Rcpp::Vector<19, Rcpp::PreserveStorage>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, int) (packages/tests-vg/crfsuite/src/rcpp_crfsuite.cpp:103)
==2295052== by 0x18236962: _crfsuite_crfsuite_model_build (packages/tests-vg/crfsuite/src/RcppExports.cpp:50)
==2295052== by 0x4A3BFD: R_doDotCall (svn/R-devel/src/main/dotcode.c:904)
==2295052== by 0x4A4203: do_dotcall (svn/R-devel/src/main/dotcode.c:1551)
==2295052== by 0x4DD026: bcEval (svn/R-devel/src/main/eval.c:7567)
==2295052== by 0x4F595F: Rf_eval (svn/R-devel/src/main/eval.c:1146)
==2295052== by 0x4F7751: R_execClosure (svn/R-devel/src/main/eval.c:2320)
==2295052==
==2295052== realloc() with size 0
==2295052== at 0x4846A40: realloc (/builddir/build/BUILD/valgrind-3.21.0/coregrind/m_replacemalloc/vg_replace_malloc.c:1649)
==2295052== by 0x1822121E: mem_mgr (packages/tests-vg/crfsuite/src/crf/src/rumavl.c:1107)
==2295052== by 0x1822121E: node_destroy (packages/tests-vg/crfsuite/src/crf/src/rumavl.c:834)
==2295052== by 0x1822121E: rumavl_destroy (packages/tests-vg/crfsuite/src/crf/src/rumavl.c:266)
==2295052== by 0x1821BAA2: featureset_delete (packages/tests-vg/crfsuite/src/crf/src/crf1d_feature.c:96)
==2295052== by 0x1821BAA2: crf1df_generate (packages/tests-vg/crfsuite/src/crf/src/crf1d_feature.c:254)
==2295052== by 0x1821B28F: crf1de_set_data (packages/tests-vg/crfsuite/src/crf/src/crf1d_encode.c:470)
==2295052== by 0x1821B28F: encoder_initialize (packages/tests-vg/crfsuite/src/crf/src/crf1d_encode.c:880)
==2295052== by 0x1821F048: crfsuite_train_train (packages/tests-vg/crfsuite/src/crf/src/crfsuite_train.c:155)
==2295052== by 0x182298E0: crfsuite_model_build(char const*, std::vector<int, std::allocator<int> >, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, Rcpp::Matrix<16, Rcpp::PreserveStorage>, Rcpp::Matrix<14, Rcpp::PreserveStorage>, Rcpp::Vector<19, Rcpp::PreserveStorage>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, int) (packages/tests-vg/crfsuite/src/rcpp_crfsuite.cpp:111)
==2295052== by 0x18236962: _crfsuite_crfsuite_model_build (packages/tests-vg/crfsuite/src/RcppExports.cpp:50)
==2295052== by 0x4A3BFD: R_doDotCall (svn/R-devel/src/main/dotcode.c:904)
==2295052== by 0x4A4203: do_dotcall (svn/R-devel/src/main/dotcode.c:1551)
==2295052== by 0x4DD026: bcEval (svn/R-devel/src/main/eval.c:7567)
==2295052== by 0x4F595F: Rf_eval (svn/R-devel/src/main/eval.c:1146)
==2295052== by 0x4F7751: R_execClosure (svn/R-devel/src/main/eval.c:2320)
==2295052== Address 0x1d4fbe90 is 0 bytes inside a block of size 24 alloc'd
==2295052== at 0x484182F: malloc (/builddir/build/BUILD/valgrind-3.21.0/coregrind/m_replacemalloc/vg_replace_malloc.c:431)
==2295052== by 0x18221085: mem_mgr (packages/tests-vg/crfsuite/src/crf/src/rumavl.c:1107)
==2295052== by 0x18221085: node_new (packages/tests-vg/crfsuite/src/crf/src/rumavl.c:815)
==2295052== by 0x182214D4: rumavl_set (packages/tests-vg/crfsuite/src/crf/src/rumavl.c:308)
==2295052== by 0x1822154C: rumavl_insert (packages/tests-vg/crfsuite/src/crf/src/rumavl.c:392)
==2295052== by 0x1821B9AA: featureset_add (packages/tests-vg/crfsuite/src/crf/src/crf1d_feature.c:107)
==2295052== by 0x1821B9AA: featureset_add (packages/tests-vg/crfsuite/src/crf/src/crf1d_feature.c:101)
==2295052== by 0x1821B9AA: crf1df_generate (packages/tests-vg/crfsuite/src/crf/src/crf1d_feature.c:212)
==2295052== by 0x1821B28F: crf1de_set_data (packages/tests-vg/crfsuite/src/crf/src/crf1d_encode.c:470)
==2295052== by 0x1821B28F: encoder_initialize (packages/tests-vg/crfsuite/src/crf/src/crf1d_encode.c:880)
==2295052== by 0x1821F048: crfsuite_train_train (packages/tests-vg/crfsuite/src/crf/src/crfsuite_train.c:155)
==2295052== by 0x182298E0: crfsuite_model_build(char const*, std::vector<int, std::allocator<int> >, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, Rcpp::Matrix<16, Rcpp::PreserveStorage>, Rcpp::Matrix<14, Rcpp::PreserveStorage>, Rcpp::Vector<19, Rcpp::PreserveStorage>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, int) (packages/tests-vg/crfsuite/src/rcpp_crfsuite.cpp:111)
==2295052== by 0x18236962: _crfsuite_crfsuite_model_build (packages/tests-vg/crfsuite/src/RcppExports.cpp:50)
==2295052== by 0x4A3BFD: R_doDotCall (svn/R-devel/src/main/dotcode.c:904)
==2295052== by 0x4A4203: do_dotcall (svn/R-devel/src/main/dotcode.c:1551)
==2295052== by 0x4DD026: bcEval (svn/R-devel/src/main/eval.c:7567)
==2295052==
==2295052== realloc() with size 0
==2295052== at 0x4846A40: realloc (/builddir/build/BUILD/valgrind-3.21.0/coregrind/m_replacemalloc/vg_replace_malloc.c:1649)
==2295052== by 0x18221232: mem_mgr (packages/tests-vg/crfsuite/src/crf/src/rumavl.c:1107)
==2295052== by 0x18221232: node_destroy (packages/tests-vg/crfsuite/src/crf/src/rumavl.c:835)
==2295052== by 0x18221232: rumavl_destroy (packages/tests-vg/crfsuite/src/crf/src/rumavl.c:266)
==2295052== by 0x1821BAA2: featureset_delete (packages/tests-vg/crfsuite/src/crf/src/crf1d_feature.c:96)
==2295052== by 0x1821BAA2: crf1df_generate (packages/tests-vg/crfsuite/src/crf/src/crf1d_feature.c:254)
==2295052== by 0x1821B28F: crf1de_set_data (packages/tests-vg/crfsuite/src/crf/src/crf1d_encode.c:470)
==2295052== by 0x1821B28F: encoder_initialize (packages/tests-vg/crfsuite/src/crf/src/crf1d_encode.c:880)
==2295052== by 0x1821F048: crfsuite_train_train (packages/tests-vg/crfsuite/src/crf/src/crfsuite_train.c:155)
==2295052== by 0x182298E0: crfsuite_model_build(char const*, std::vector<int, std::allocator<int> >, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, Rcpp::Matrix<16, Rcpp::PreserveStorage>, Rcpp::Matrix<14, Rcpp::PreserveStorage>, Rcpp::Vector<19, Rcpp::PreserveStorage>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, int) (packages/tests-vg/crfsuite/src/rcpp_crfsuite.cpp:111)
==2295052== by 0x18236962: _crfsuite_crfsuite_model_build (packages/tests-vg/crfsuite/src/RcppExports.cpp:50)
==2295052== by 0x4A3BFD: R_doDotCall (svn/R-devel/src/main/dotcode.c:904)
==2295052== by 0x4A4203: do_dotcall (svn/R-devel/src/main/dotcode.c:1551)
==2295052== by 0x4DD026: bcEval (svn/R-devel/src/main/eval.c:7567)
==2295052== by 0x4F595F: Rf_eval (svn/R-devel/src/main/eval.c:1146)
==2295052== by 0x4F7751: R_execClosure (svn/R-devel/src/main/eval.c:2320)
==2295052== Address 0x1d4fbe30 is 0 bytes inside a block of size 32 alloc'd
==2295052== at 0x484182F: malloc (/builddir/build/BUILD/valgrind-3.21.0/coregrind/m_replacemalloc/vg_replace_malloc.c:431)
==2295052== by 0x18221071: mem_mgr (packages/tests-vg/crfsuite/src/crf/src/rumavl.c:1107)
==2295052== by 0x18221071: node_new (packages/tests-vg/crfsuite/src/crf/src/rumavl.c:812)
==2295052== by 0x182214D4: rumavl_set (packages/tests-vg/crfsuite/src/crf/src/rumavl.c:308)
==2295052== by 0x1822154C: rumavl_insert (packages/tests-vg/crfsuite/src/crf/src/rumavl.c:392)
==2295052== by 0x1821B9AA: featureset_add (packages/tests-vg/crfsuite/src/crf/src/crf1d_feature.c:107)
==2295052== by 0x1821B9AA: featureset_add (packages/tests-vg/crfsuite/src/crf/src/crf1d_feature.c:101)
==2295052== by 0x1821B9AA: crf1df_generate (packages/tests-vg/crfsuite/src/crf/src/crf1d_feature.c:212)
==2295052== by 0x1821B28F: crf1de_set_data (packages/tests-vg/crfsuite/src/crf/src/crf1d_encode.c:470)
==2295052== by 0x1821B28F: encoder_initialize (packages/tests-vg/crfsuite/src/crf/src/crf1d_encode.c:880)
==2295052== by 0x1821F048: crfsuite_train_train (packages/tests-vg/crfsuite/src/crf/src/crfsuite_train.c:155)
==2295052== by 0x182298E0: crfsuite_model_build(char const*, std::vector<int, std::allocator<int> >, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, Rcpp::Matrix<16, Rcpp::PreserveStorage>, Rcpp::Matrix<14, Rcpp::PreserveStorage>, Rcpp::Vector<19, Rcpp::PreserveStorage>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, int) (packages/tests-vg/crfsuite/src/rcpp_crfsuite.cpp:111)
==2295052== by 0x18236962: _crfsuite_crfsuite_model_build (packages/tests-vg/crfsuite/src/RcppExports.cpp:50)
==2295052== by 0x4A3BFD: R_doDotCall (svn/R-devel/src/main/dotcode.c:904)
==2295052== by 0x4A4203: do_dotcall (svn/R-devel/src/main/dotcode.c:1551)
==2295052== by 0x4DD026: bcEval (svn/R-devel/src/main/eval.c:7567)
==2295052==
==2295052== realloc() with size 0
==2295052== at 0x4846A40: realloc (/builddir/build/BUILD/valgrind-3.21.0/coregrind/m_replacemalloc/vg_replace_malloc.c:1649)
==2295052== by 0x1822121E: mem_mgr (packages/tests-vg/crfsuite/src/crf/src/rumavl.c:1107)
==2295052== by 0x1822121E: node_destroy (packages/tests-vg/crfsuite/src/crf/src/rumavl.c:834)
==2295052== by 0x1822121E: rumavl_destroy (packages/tests-vg/crfsuite/src/crf/src/rumavl.c:266)
==2295052== by 0x18220981: quark_delete (packages/tests-vg/crfsuite/src/crf/src/quark.c:91)
==2295052== by 0x1821F8AB: dictionary_release (packages/tests-vg/crfsuite/src/crf/src/dictionary.c:51)
==2295052== by 0x1821F8AB: dictionary_release (packages/tests-vg/crfsuite/src/crf/src/dictionary.c:46)
==2295052== by 0x1822582B: CRFSuite::Trainer::clear() (packages/tests-vg/crfsuite/src/./include/crfsuite.hpp:95)
==2295052== by 0x1822588D: CRFSuite::Trainer::~Trainer() (packages/tests-vg/crfsuite/src/./include/crfsuite.hpp:62)
==2295052== by 0x1822A341: crfsuite_model_build(char const*, std::vector<int, std::allocator<int> >, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, Rcpp::Matrix<16, Rcpp::PreserveStorage>, Rcpp::Matrix<14, Rcpp::PreserveStorage>, Rcpp::Vector<19, Rcpp::PreserveStorage>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, int) (packages/tests-vg/crfsuite/src/rcpp_crfsuite.cpp:137)
==2295052== by 0x18236962: _crfsuite_crfsuite_model_build (packages/tests-vg/crfsuite/src/RcppExports.cpp:50)
==2295052== by 0x4A3BFD: R_doDotCall (svn/R-devel/src/main/dotcode.c:904)
==2295052== by 0x4A4203: do_dotcall (svn/R-devel/src/main/dotcode.c:1551)
==2295052== by 0x4DD026: bcEval (svn/R-devel/src/main/eval.c:7567)
==2295052== by 0x4F595F: Rf_eval (svn/R-devel/src/main/eval.c:1146)
==2295052== Address 0x1cb31ec0 is 0 bytes inside a block of size 16 alloc'd
==2295052== at 0x484182F: malloc (/builddir/build/BUILD/valgrind-3.21.0/coregrind/m_replacemalloc/vg_replace_malloc.c:431)
==2295052== by 0x18221085: mem_mgr (packages/tests-vg/crfsuite/src/crf/src/rumavl.c:1107)
==2295052== by 0x18221085: node_new (packages/tests-vg/crfsuite/src/crf/src/rumavl.c:815)
==2295052== by 0x18221452: rumavl_set (packages/tests-vg/crfsuite/src/crf/src/rumavl.c:358)
==2295052== by 0x1822154C: rumavl_insert (packages/tests-vg/crfsuite/src/crf/src/rumavl.c:392)
==2295052== by 0x18220A31: quark_get (packages/tests-vg/crfsuite/src/crf/src/quark.c:117)
==2295052== by 0x1822657F: CRFSuite::Trainer::append(std::vector<std::vector<CRFSuite::Attribute, std::allocator<CRFSuite::Attribute> >, std::allocator<std::vector<CRFSuite::Attribute, std::allocator<CRFSuite::Attribute> > > > const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, int) (packages/tests-vg/crfsuite/src/./include/crfsuite.hpp:138)
==2295052== by 0x18229756: crfsuite_model_build(char const*, std::vector<int, std::allocator<int> >, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, Rcpp::Matrix<16, Rcpp::PreserveStorage>, Rcpp::Matrix<14, Rcpp::PreserveStorage>, Rcpp::Vector<19, Rcpp::PreserveStorage>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, int) (packages/tests-vg/crfsuite/src/rcpp_crfsuite.cpp:103)
==2295052== by 0x18236962: _crfsuite_crfsuite_model_build (packages/tests-vg/crfsuite/src/RcppExports.cpp:50)
==2295052== by 0x4A3BFD: R_doDotCall (svn/R-devel/src/main/dotcode.c:904)
==2295052== by 0x4A4203: do_dotcall (svn/R-devel/src/main/dotcode.c:1551)
==2295052== by 0x4DD026: bcEval (svn/R-devel/src/main/eval.c:7567)
==2295052== by 0x4F595F: Rf_eval (svn/R-devel/src/main/eval.c:1146)
==2295052==
==2295052== realloc() with size 0
==2295052== at 0x4846A40: realloc (/builddir/build/BUILD/valgrind-3.21.0/coregrind/m_replacemalloc/vg_replace_malloc.c:1649)
==2295052== by 0x18221232: mem_mgr (packages/tests-vg/crfsuite/src/crf/src/rumavl.c:1107)
==2295052== by 0x18221232: node_destroy (packages/tests-vg/crfsuite/src/crf/src/rumavl.c:835)
==2295052== by 0x18221232: rumavl_destroy (packages/tests-vg/crfsuite/src/crf/src/rumavl.c:266)
==2295052== by 0x18220981: quark_delete (packages/tests-vg/crfsuite/src/crf/src/quark.c:91)
==2295052== by 0x1821F8AB: dictionary_release (packages/tests-vg/crfsuite/src/crf/src/dictionary.c:51)
==2295052== by 0x1821F8AB: dictionary_release (packages/tests-vg/crfsuite/src/crf/src/dictionary.c:46)
==2295052== by 0x1822582B: CRFSuite::Trainer::clear() (packages/tests-vg/crfsuite/src/./include/crfsuite.hpp:95)
==2295052== by 0x1822588D: CRFSuite::Trainer::~Trainer() (packages/tests-vg/crfsuite/src/./include/crfsuite.hpp:62)
==2295052== by 0x1822A341: crfsuite_model_build(char const*, std::vector<int, std::allocator<int> >, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, Rcpp::Matrix<16, Rcpp::PreserveStorage>, Rcpp::Matrix<14, Rcpp::PreserveStorage>, Rcpp::Vector<19, Rcpp::PreserveStorage>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, int) (packages/tests-vg/crfsuite/src/rcpp_crfsuite.cpp:137)
==2295052== by 0x18236962: _crfsuite_crfsuite_model_build (packages/tests-vg/crfsuite/src/RcppExports.cpp:50)
==2295052== by 0x4A3BFD: R_doDotCall (svn/R-devel/src/main/dotcode.c:904)
==2295052== by 0x4A4203: do_dotcall (svn/R-devel/src/main/dotcode.c:1551)
==2295052== by 0x4DD026: bcEval (svn/R-devel/src/main/eval.c:7567)
==2295052== by 0x4F595F: Rf_eval (svn/R-devel/src/main/eval.c:1146)
==2295052== Address 0x1cb31e60 is 0 bytes inside a block of size 32 alloc'd
==2295052== at 0x484182F: malloc (/builddir/build/BUILD/valgrind-3.21.0/coregrind/m_replacemalloc/vg_replace_malloc.c:431)
==2295052== by 0x18221071: mem_mgr (packages/tests-vg/crfsuite/src/crf/src/rumavl.c:1107)
==2295052== by 0x18221071: node_new (packages/tests-vg/crfsuite/src/crf/src/rumavl.c:812)
==2295052== by 0x18221452: rumavl_set (packages/tests-vg/crfsuite/src/crf/src/rumavl.c:358)
==2295052== by 0x1822154C: rumavl_insert (packages/tests-vg/crfsuite/src/crf/src/rumavl.c:392)
==2295052== by 0x18220A31: quark_get (packages/tests-vg/crfsuite/src/crf/src/quark.c:117)
==2295052== by 0x1822657F: CRFSuite::Trainer::append(std::vector<std::vector<CRFSuite::Attribute, std::allocator<CRFSuite::Attribute> >, std::allocator<std::vector<CRFSuite::Attribute, std::allocator<CRFSuite::Attribute> > > > const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, int) (packages/tests-vg/crfsuite/src/./include/crfsuite.hpp:138)
==2295052== by 0x18229756: crfsuite_model_build(char const*, std::vector<int, std::allocator<int> >, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, Rcpp::Matrix<16, Rcpp::PreserveStorage>, Rcpp::Matrix<14, Rcpp::PreserveStorage>, Rcpp::Vector<19, Rcpp::PreserveStorage>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, int) (packages/tests-vg/crfsuite/src/rcpp_crfsuite.cpp:103)
==2295052== by 0x18236962: _crfsuite_crfsuite_model_build (packages/tests-vg/crfsuite/src/RcppExports.cpp:50)
==2295052== by 0x4A3BFD: R_doDotCall (svn/R-devel/src/main/dotcode.c:904)
==2295052== by 0x4A4203: do_dotcall (svn/R-devel/src/main/dotcode.c:1551)
==2295052== by 0x4DD026: bcEval (svn/R-devel/src/main/eval.c:7567)
==2295052== by 0x4F595F: Rf_eval (svn/R-devel/src/main/eval.c:1146)
==2295052==
Summary statistics of last iteration:
Loss: 2999.873030
Feature norm: 3.410088
Error norm: 742.128202
Active features: 310
Line search trials: 2
Line search step: 0.039598
Seconds required for this iteration: 0.457
Dumping summary of the model to file /tmp/RtmpRrKCfR/crfsuite_23050c535654bd.txt
Copying summary of the model to file modeldetails.txt in folder /data/blackswan/ripley/R/packages/tests-vg/crfsuite.Rcheck
[1] TRUE
>
>
>
> cleanEx()
> nameEx("crf_cbind_attributes")
> ### * crf_cbind_attributes
>
> flush(stderr()); flush(stdout())
>
> ### Name: crf_cbind_attributes
> ### Title: Enrich a data.frame by adding frequently used CRF attributes
> ### Aliases: crf_cbind_attributes
>
> ### ** Examples
>
> x <- data.frame(doc_id = sort(sample.int(n = 10, size = 1000, replace = TRUE)))
> x$pos <- sample(c("Art", "N", "Prep", "V", "Adv", "Adj", "Conj",
+ "Punc", "Num", "Pron", "Int", "Misc"),
+ size = nrow(x), replace = TRUE)
> x <- crf_cbind_attributes(x, terms = "pos", by = "doc_id",
+ from = -1, to = 1, ngram_max = 3)
> head(x)
doc_id pos pos[t-1] pos[t] pos[t+1] pos[t-1]-pos[t]
1 1 Pron <NA> pos[t]=Pron pos[t+1]=Prep <NA>
2 1 Prep pos[t-1]=Pron pos[t]=Prep pos[t+1]=Conj pos[t-1]-pos[t]=Pron-Prep
3 1 Conj pos[t-1]=Prep pos[t]=Conj pos[t+1]=Num pos[t-1]-pos[t]=Prep-Conj
4 1 Num pos[t-1]=Conj pos[t]=Num pos[t+1]=Num pos[t-1]-pos[t]=Conj-Num
5 1 Num pos[t-1]=Num pos[t]=Num pos[t+1]=Prep pos[t-1]-pos[t]=Num-Num
6 1 Prep pos[t-1]=Num pos[t]=Prep pos[t+1]=Conj pos[t-1]-pos[t]=Num-Prep
pos[t-1]-pos[t+1] pos[t]-pos[t+1]
1 <NA> pos[t]-pos[t+1]=Pron-Prep
2 pos[t-1]-pos[t+1]=Pron-Conj pos[t]-pos[t+1]=Prep-Conj
3 pos[t-1]-pos[t+1]=Prep-Num pos[t]-pos[t+1]=Conj-Num
4 pos[t-1]-pos[t+1]=Conj-Num pos[t]-pos[t+1]=Num-Num
5 pos[t-1]-pos[t+1]=Num-Prep pos[t]-pos[t+1]=Num-Prep
6 pos[t-1]-pos[t+1]=Num-Conj pos[t]-pos[t+1]=Prep-Conj
pos[t-1]-pos[t]-pos[t+1]
1 <NA>
2 pos[t-1]-pos[t]-pos[t+1]=Pron-Prep-Conj
3 pos[t-1]-pos[t]-pos[t+1]=Prep-Conj-Num
4 pos[t-1]-pos[t]-pos[t+1]=Conj-Num-Num
5 pos[t-1]-pos[t]-pos[t+1]=Num-Num-Prep
6 pos[t-1]-pos[t]-pos[t+1]=Num-Prep-Conj
>
>
>
>
> cleanEx()
> nameEx("crf_evaluation")
> ### * crf_evaluation
>
> flush(stderr()); flush(stdout())
>
> ### Name: crf_evaluation
> ### Title: Basic classification evaluation metrics for multi-class
> ### labelling
> ### Aliases: crf_evaluation
>
> ### ** Examples
>
> pred <- sample(LETTERS, 1000, replace = TRUE)
> gold <- sample(LETTERS, 1000, replace = TRUE)
> crf_evaluation(pred = pred, obs = gold, labels = LETTERS)
$bylabel
label accuracy precision recall specificity f1 support
1 A 0.934 0.05000000 0.06666667 0.9608247 0.05714286 30
2 B 0.921 0.02325581 0.02631579 0.9563410 0.02469136 38
3 C 0.925 0.02702703 0.02500000 0.9625000 0.02597403 40
4 D 0.932 0.03225806 0.02564103 0.9687825 0.02857143 39
5 E 0.922 0.00000000 0.00000000 0.9554404 0.00000000 35
6 F 0.929 0.00000000 0.00000000 0.9577320 0.00000000 30
7 G 0.928 0.02777778 0.02631579 0.9636175 0.02702703 38
8 H 0.927 0.02500000 0.02857143 0.9595855 0.02666667 35
9 I 0.919 0.07500000 0.06382979 0.9611752 0.06896552 47
10 J 0.927 0.08333333 0.06976744 0.9655172 0.07594937 43
11 K 0.934 0.07142857 0.04761905 0.9728601 0.05714286 42
12 L 0.924 0.02380952 0.02777778 0.9574689 0.02564103 36
13 M 0.927 0.05000000 0.05405405 0.9605400 0.05194805 37
14 N 0.927 0.08333333 0.06976744 0.9655172 0.07594937 43
15 O 0.928 0.03125000 0.02380952 0.9676409 0.02702703 42
16 P 0.920 0.00000000 0.00000000 0.9603340 0.00000000 42
17 Q 0.941 0.03225806 0.03333333 0.9690722 0.03278689 30
18 R 0.947 0.08000000 0.06250000 0.9762397 0.07017544 32
19 S 0.928 0.02222222 0.03448276 0.9546859 0.02702703 29
20 T 0.923 0.11363636 0.11627907 0.9592476 0.11494253 43
21 U 0.922 0.02222222 0.02857143 0.9544041 0.02500000 35
22 V 0.915 0.00000000 0.00000000 0.9561129 0.00000000 43
23 W 0.922 0.10000000 0.08695652 0.9622642 0.09302326 46
24 X 0.927 0.00000000 0.00000000 0.9656250 0.00000000 40
25 Y 0.912 0.02272727 0.02173913 0.9549266 0.02222222 46
26 Z 0.917 0.04166667 0.05128205 0.9521332 0.04597701 39
$overall
accuracy precision recall specificity
0.03900000 0.04191964 0.03966597 0.96132319
f1 precision_mean recall_mean specificity_mean
0.04040634 0.04027825 0.03865882 0.96131790
f1_mean
0.03907296
>
>
>
>
> cleanEx()
> nameEx("crf_options")
> ### * crf_options
>
> flush(stderr()); flush(stdout())
>
> ### Name: crf_options
> ### Title: Conditional Random Fields parameters
> ### Aliases: crf_options
>
> ### ** Examples
>
> # L-BFGS with L1/L2 regularization
> opts <- crf_options("lbfgs")
> str(opts)
List of 4
$ method : chr "lbfgs"
$ type : chr "crf1d"
$ params :'data.frame': 12 obs. of 3 variables:
..$ arg : chr [1:12] "feature.minfreq" "feature.possible_states" "feature.possible_transitions" "c1" ...
..$ arg_default: chr [1:12] "0.000000" "0" "0" "0.000000" ...
..$ description: chr [1:12] "The minimum frequency of features." "Force to generate possible state features." "Force to generate possible transition features." "Coefficient for L1 regularization." ...
$ default:List of 12
..$ feature.minfreq : num 0
..$ feature.possible_states : num 0
..$ feature.possible_transitions: num 0
..$ c1 : num 0
..$ c2 : num 1
..$ max_iterations : num 2.15e+09
..$ num_memories : num 6
..$ epsilon : num 1e-05
..$ period : num 10
..$ delta : num 1e-05
..$ linesearch : chr "MoreThuente"
..$ max_linesearch : num 20
>
> # SGD with L2-regularization
> crf_options("l2sgd")
$method
[1] "l2sgd"
$type
[1] "crf1d"
$params
arg arg_default
1 feature.minfreq 0.000000
2 feature.possible_states 0
3 feature.possible_transitions 0
4 c2 1.000000
5 max_iterations 1000
6 period 10
7 delta 0.000001
8 calibration.eta 0.100000
9 calibration.rate 2.000000
10 calibration.samples 1000
11 calibration.candidates 10
12 calibration.max_trials 20
description
1 The minimum frequency of features.
2 Force to generate possible state features.
3 Force to generate possible transition features.
4 Coefficient for L2 regularization.
5 The maximum number of iterations (epochs) for SGD optimization.
6 The duration of iterations to test the stopping criterion.
7 The threshold for the stopping criterion; an optimization process stops when\nthe improvement of the log likelihood over the last ${period} iterations is no\ngreater than this threshold.
8 The initial value of learning rate (eta) used for calibration.
9 The rate of increase/decrease of learning rate for calibration.
10 The number of instances used for calibration.
11 The number of candidates of learning rate.
12 The maximum number of trials of learning rates for calibration.
$default
$default$feature.minfreq
[1] 0
$default$feature.possible_states
[1] 0
$default$feature.possible_transitions
[1] 0
$default$c2
[1] 1
$default$max_iterations
[1] 1000
$default$period
[1] 10
$default$delta
[1] 1e-06
$default$calibration.eta
[1] 0.1
$default$calibration.rate
[1] 2
$default$calibration.samples
[1] 1000
$default$calibration.candidates
[1] 10
$default$calibration.max_trials
[1] 20
>
> # Averaged Perceptron
> crf_options("averaged-perceptron")
$method
[1] "averaged-perceptron"
$type
[1] "crf1d"
$params
arg arg_default
1 feature.minfreq 0.000000
2 feature.possible_states 0
3 feature.possible_transitions 0
4 max_iterations 100
5 epsilon 0.000000
description
1 The minimum frequency of features.
2 Force to generate possible state features.
3 Force to generate possible transition features.
4 The maximum number of iterations.
5 The stopping criterion (the ratio of incorrect label predictions).
$default
$default$feature.minfreq
[1] 0
$default$feature.possible_states
[1] 0
$default$feature.possible_transitions
[1] 0
$default$max_iterations
[1] 100
$default$epsilon
[1] 0
>
> # Passive Aggressive
> crf_options("passive-aggressive")
$method
[1] "passive-aggressive"
$type
[1] "crf1d"
$params
arg arg_default
1 feature.minfreq 0.000000
2 feature.possible_states 0
3 feature.possible_transitions 0
4 type 1
5 c 1.000000
6 error_sensitive 1
7 averaging 1
8 max_iterations 100
9 epsilon 0.000000
description
1 The minimum frequency of features.
2 Force to generate possible state features.
3 Force to generate possible transition features.
4 The strategy for updating feature weights: {\n 0: PA without slack variables,\n 1: PA type I,\n 2: PA type II\n}.\n
5 The aggressiveness parameter.
6 Consider the number of incorrect labels to the cost function.
7 Compute the average of feature weights (similarly to Averaged Perceptron).
8 The maximum number of iterations.
9 The stopping criterion (the mean loss).
$default
$default$feature.minfreq
[1] 0
$default$feature.possible_states
[1] 0
$default$feature.possible_transitions
[1] 0
$default$type
[1] 1
$default$c
[1] 1
$default$error_sensitive
[1] 1
$default$averaging
[1] 1
$default$max_iterations
[1] 100
$default$epsilon
[1] 0
>
> # Adaptive Regularization of Weights (AROW)
> crf_options("arow")
$method
[1] "arow"
$type
[1] "crf1d"
$params
arg arg_default
1 feature.minfreq 0.000000
2 feature.possible_states 0
3 feature.possible_transitions 0
4 variance 1.000000
5 gamma 1.000000
6 max_iterations 100
7 epsilon 0.000000
description
1 The minimum frequency of features.
2 Force to generate possible state features.
3 Force to generate possible transition features.
4 The initial variance of every feature weight.
5 Tradeoff parameter.
6 The maximum number of iterations.
7 The stopping criterion (the mean loss).
$default
$default$feature.minfreq
[1] 0
$default$feature.possible_states
[1] 0
$default$feature.possible_transitions
[1] 0
$default$variance
[1] 1
$default$gamma
[1] 1
$default$max_iterations
[1] 100
$default$epsilon
[1] 0
>
>
>
> cleanEx()
> nameEx("merge.chunkrange")
> ### * merge.chunkrange
>
> flush(stderr()); flush(stdout())
>
> ### Name: merge.chunkrange
> ### Title: CRF Training data construction: add chunk entity category to a
> ### tokenised dataset
> ### Aliases: merge.chunkrange
>
> ### ** Examples
>
>
>
>
> cleanEx()
> nameEx("ner_download_modeldata")
> ### * ner_download_modeldata
>
> flush(stderr()); flush(stdout())
>
> ### Name: ner_download_modeldata
> ### Title: CRF Training data: download training data for doing Named Entity
> ### Recognition (NER)
> ### Aliases: ner_download_modeldata
>
> ### ** Examples
>
> ## Not run:
> ##D x <- ner_download_modeldata("conll2002-nl")
> ##D x <- ner_download_modeldata("conll2002-es")
> ##D x <- ner_download_modeldata("GermanNER")
> ##D x <- ner_download_modeldata("wikiner-en-wp2")
> ##D x <- ner_download_modeldata("wikiner-nl-wp3")
> ##D x <- ner_download_modeldata("wikiner-fr-wp3")
> ## End(Not run)
> ## reduce number of docs
> x <- ner_download_modeldata("conll2002-es", docs = 10)
trying URL 'https://raw.githubusercontent.com/bnosac-dev/nltk_data/gh-pages/packages/corpora/conll2002.zip'
Content type 'application/zip' length 1867449 bytes (1.8 MB)
==================================================
downloaded 1.8 MB
>
>
>
> cleanEx()
> nameEx("predict.crf")
> ### * predict.crf
>
> flush(stderr()); flush(stdout())
>
> ### Name: predict.crf
> ### Title: Predict the label sequence based on the Conditional Random Field
> ### Aliases: predict.crf
>
> ### ** Examples
>
>
>
>
> cleanEx()
> nameEx("txt_feature")
> ### * txt_feature
>
> flush(stderr()); flush(stdout())
>
> ### Name: txt_feature
> ### Title: Extract basic text features which are useful for entity
> ### recognition
> ### Aliases: txt_feature
>
> ### ** Examples
>
> txt_feature("Red Devils", type = "is_capitalised")
[1] TRUE
> txt_feature("red devils", type = "is_capitalised")
[1] FALSE
> txt_feature("http://www.bnosac.be", type = "is_url")
[1] TRUE
> txt_feature("[email protected]", type = "is_email")
[1] TRUE
> txt_feature("hi there", type = "is_email")
[1] FALSE
> txt_feature("1230000", type = "is_number")
[1] TRUE
> txt_feature("123.15", type = "is_number")
[1] TRUE
> txt_feature("123,15", type = "is_number")
[1] TRUE
> txt_feature("123abc", type = "is_number")
[1] FALSE
> txt_feature("abcdefghijklmnopqrstuvwxyz", type = "prefix", n = 3)
[1] "abc"
> txt_feature("abcdefghijklmnopqrstuvwxyz", type = "suffix", n = 3)
[1] "xyz"
> txt_feature("Red Devils", type = "shape")
[1] "Xxx Xxxxxx"
> txt_feature("red devils", type = "shape")
[1] "xxx xxxxxx"
>
>
>
> cleanEx()
> nameEx("txt_sprintf")
> ### * txt_sprintf
>
> flush(stderr()); flush(stdout())
>
> ### Name: txt_sprintf
> ### Title: 'NA' friendly version of sprintf
> ### Aliases: txt_sprintf
>
> ### ** Examples
>
> sprintf("(w-1):%s", c("xyz", NA, "abc"))
[1] "(w-1):xyz" "(w-1):NA" "(w-1):abc"
> txt_sprintf("(w-1):%s", c("xyz", NA, "abc"))
[1] "(w-1):xyz" NA "(w-1):abc"
> sprintf("(w-1):%s_%s", c("xyz", NA, "abc"), c(NA, "xyz", "abc"))
[1] "(w-1):xyz_NA" "(w-1):NA_xyz" "(w-1):abc_abc"
> txt_sprintf("(w-1):%s_%s", c("xyz", NA, "abc"), c(NA, "xyz", "abc"))
[1] NA NA "(w-1):abc_abc"
>
>
>
> ### * <FOOTER>
> ###
> cleanEx()
> options(digits = 7L)
> base::cat("Time elapsed: ", proc.time() - base::get("ptime", pos = 'CheckExEnv'),"\n")
Time elapsed: 177.881 1.238 181.327 0.158 0.603
> grDevices::dev.off()
null device
1
> ###
> ### Local variables: ***
> ### mode: outline-minor ***
> ### outline-regexp: "\\(> \\)?### [*]+" ***
> ### End: ***
> quit('no')
==2295052==
==2295052== HEAP SUMMARY:
==2295052== in use at exit: 131,689,871 bytes in 17,365 blocks
==2295052== total heap usage: 833,540 allocs, 816,175 frees, 293,583,458 bytes allocated
==2295052==
==2295052== 384 bytes in 1 blocks are possibly lost in loss record 225 of 2,181
==2295052== at 0x484682C: calloc (/builddir/build/BUILD/valgrind-3.21.0/coregrind/m_replacemalloc/vg_replace_malloc.c:1554)
==2295052== by 0x4011B93: UnknownInlinedFun (/usr/src/debug/glibc-2.37-4.fc38.x86_64/elf/../include/rtld-malloc.h:44)
==2295052== by 0x4011B93: allocate_dtv (/usr/src/debug/glibc-2.37-4.fc38.x86_64/elf/../elf/dl-tls.c:375)
==2295052== by 0x4012601: _dl_allocate_tls (/usr/src/debug/glibc-2.37-4.fc38.x86_64/elf/../elf/dl-tls.c:634)
==2295052== by 0x54714D5: allocate_stack (/usr/src/debug/glibc-2.37-4.fc38.x86_64/nptl/allocatestack.c:423)
==2295052== by 0x54714D5: pthread_create@@GLIBC_2.34 (/usr/src/debug/glibc-2.37-4.fc38.x86_64/nptl/pthread_create.c:652)
==2295052== by 0x187CD65E: UnknownInlinedFun (/usr/src/debug/curl-8.0.1-3.fc38.x86_64/build-full/lib/../../lib/curl_threads.c:72)
==2295052== by 0x187CD65E: UnknownInlinedFun (/usr/src/debug/curl-8.0.1-3.fc38.x86_64/build-full/lib/../../lib/asyn-thread.c:458)
==2295052== by 0x187CD65E: UnknownInlinedFun (/usr/src/debug/curl-8.0.1-3.fc38.x86_64/build-full/lib/../../lib/asyn-thread.c:711)
==2295052== by 0x187CD65E: UnknownInlinedFun (/usr/src/debug/curl-8.0.1-3.fc38.x86_64/build-full/lib/../../lib/hostasyn.c:120)
==2295052== by 0x187CD65E: Curl_resolv (/usr/src/debug/curl-8.0.1-3.fc38.x86_64/build-full/lib/../../lib/hostip.c:778)
==2295052== by 0x1880B7E0: UnknownInlinedFun (/usr/src/debug/curl-8.0.1-3.fc38.x86_64/build-full/lib/../../lib/hostip.c:953)
==2295052== by 0x1880B7E0: UnknownInlinedFun (/usr/src/debug/curl-8.0.1-3.fc38.x86_64/build-full/lib/../../lib/url.c:3218)
==2295052== by 0x1880B7E0: resolve_fresh.lto_priv.0 (/usr/src/debug/curl-8.0.1-3.fc38.x86_64/build-full/lib/../../lib/url.c:3263)
==2295052== by 0x187EE0F7: UnknownInlinedFun (/usr/src/debug/curl-8.0.1-3.fc38.x86_64/build-full/lib/../../lib/url.c:3285)
==2295052== by 0x187EE0F7: UnknownInlinedFun (/usr/src/debug/curl-8.0.1-3.fc38.x86_64/build-full/lib/../../lib/url.c:3830)
==2295052== by 0x187EE0F7: UnknownInlinedFun (/usr/src/debug/curl-8.0.1-3.fc38.x86_64/build-full/lib/../../lib/url.c:3901)
==2295052== by 0x187EE0F7: multi_runsingle (/usr/src/debug/curl-8.0.1-3.fc38.x86_64/build-full/lib/../../lib/multi.c:1965)
==2295052== by 0x187EE821: curl_multi_perform (/usr/src/debug/curl-8.0.1-3.fc38.x86_64/build-full/lib/../../lib/multi.c:2729)
==2295052== by 0x18651FDE: in_do_curlDownload (svn/R-devel/src/modules/internet/libcurl.c:669)
==2295052== by 0x4DD026: bcEval (svn/R-devel/src/main/eval.c:7567)
==2295052== by 0x4F595F: Rf_eval (svn/R-devel/src/main/eval.c:1146)
==2295052== by 0x4F7751: R_execClosure (svn/R-devel/src/main/eval.c:2320)
==2295052==
==2295052== 1,714 (128 direct, 1,586 indirect) bytes in 1 blocks are definitely lost in loss record 314 of 2,181
==2295052== at 0x484682C: calloc (/builddir/build/BUILD/valgrind-3.21.0/coregrind/m_replacemalloc/vg_replace_malloc.c:1554)
==2295052== by 0x182204BF: params_create_instance (packages/tests-vg/crfsuite/src/crf/src/params.c:254)
==2295052== by 0x1821F43B: crfsuite_train_new (packages/tests-vg/crfsuite/src/crf/src/crfsuite_train.c:53)
==2295052== by 0x1821F43B: crf1de_create_instance (packages/tests-vg/crfsuite/src/crf/src/crfsuite_train.c:263)
==2295052== by 0x1821E3E0: crfsuite_create_instance (packages/tests-vg/crfsuite/src/crf/src/crfsuite.c:51)
==2295052== by 0x182259C8: CRFSuite::Trainer::select(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (packages/tests-vg/crfsuite/src/./include/crfsuite.hpp:166)
==2295052== by 0x18228F3D: crfsuite_model_build(char const*, std::vector<int, std::allocator<int> >, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, Rcpp::Matrix<16, Rcpp::PreserveStorage>, Rcpp::Matrix<14, Rcpp::PreserveStorage>, Rcpp::Vector<19, Rcpp::PreserveStorage>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, int) (packages/tests-vg/crfsuite/src/rcpp_crfsuite.cpp:53)
==2295052== by 0x18236962: _crfsuite_crfsuite_model_build (packages/tests-vg/crfsuite/src/RcppExports.cpp:50)
==2295052== by 0x4A3BFD: R_doDotCall (svn/R-devel/src/main/dotcode.c:904)
==2295052== by 0x4A4203: do_dotcall (svn/R-devel/src/main/dotcode.c:1551)
==2295052== by 0x4DD026: bcEval (svn/R-devel/src/main/eval.c:7567)
==2295052== by 0x4F595F: Rf_eval (svn/R-devel/src/main/eval.c:1146)
==2295052== by 0x4F7751: R_execClosure (svn/R-devel/src/main/eval.c:2320)
==2295052==
==2295052== 5,862 (640 direct, 5,222 indirect) bytes in 5 blocks are definitely lost in loss record 369 of 2,181
==2295052== at 0x484682C: calloc (/builddir/build/BUILD/valgrind-3.21.0/coregrind/m_replacemalloc/vg_replace_malloc.c:1554)
==2295052== by 0x182204BF: params_create_instance (packages/tests-vg/crfsuite/src/crf/src/params.c:254)
==2295052== by 0x1821F43B: crfsuite_train_new (packages/tests-vg/crfsuite/src/crf/src/crfsuite_train.c:53)
==2295052== by 0x1821F43B: crf1de_create_instance (packages/tests-vg/crfsuite/src/crf/src/crfsuite_train.c:263)
==2295052== by 0x1821E3E0: crfsuite_create_instance (packages/tests-vg/crfsuite/src/crf/src/crfsuite.c:51)
==2295052== by 0x182259C8: CRFSuite::Trainer::select(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (packages/tests-vg/crfsuite/src/./include/crfsuite.hpp:166)
==2295052== by 0x182283C1: crfsuite_model_parameters(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) (packages/tests-vg/crfsuite/src/rcpp_crfsuite.cpp:15)
==2295052== by 0x18234DDA: _crfsuite_crfsuite_model_parameters (packages/tests-vg/crfsuite/src/RcppExports.cpp:31)
==2295052== by 0x4A3D09: R_doDotCall (svn/R-devel/src/main/dotcode.c:871)
==2295052== by 0x4A4203: do_dotcall (svn/R-devel/src/main/dotcode.c:1551)
==2295052== by 0x4DD026: bcEval (svn/R-devel/src/main/eval.c:7567)
==2295052== by 0x4F595F: Rf_eval (svn/R-devel/src/main/eval.c:1146)
==2295052== by 0x4F7751: R_execClosure (svn/R-devel/src/main/eval.c:2320)
==2295052==
==2295052== LEAK SUMMARY:
==2295052== definitely lost: 768 bytes in 6 blocks
==2295052== indirectly lost: 6,808 bytes in 128 blocks
==2295052== possibly lost: 384 bytes in 1 blocks
==2295052== still reachable: 131,681,911 bytes in 17,230 blocks
==2295052== suppressed: 0 bytes in 0 blocks
==2295052== Reachable blocks (those to which a pointer was found) are not shown.
==2295052== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==2295052==
==2295052== For lists of detected and suppressed errors, rerun with: -s
==2295052== ERROR SUMMARY: 71094 errors from 9 contexts (suppressed: 0 from 0)
from crfsuite.
jwijffels
commented on May 26, 2024
Fixed with f213d0e and 79ae159
from crfsuite.
Related Issues (20)
- Add crf attribute neighbouring features, integrate udpipe, add gui
- as.crf
- need a better way of keeping the transformation function from raw text to enriched crf features
- meaning of min_freq parameter HOT 4
- Tutorial HOT 16
- vignette crfsuite-nlp not found HOT 10
- Using other models than UDPipe HOT 3
- bug app newline: https://stackoverflow.com/questions/38364874/preserve-newlines-on-user-selected-text-currently-trying-with-window-getselecti HOT 2
- Difference in start & end position between shiny app and udpipe() HOT 3
- Intent Classification HOT 3
- allow to pass on an embedding matrix HOT 1
- make examples / vignette conditionally on package availability
- crf/src/crf1d_model.c:127:12: warning: writing 16 bytes into a region of size 4 [-Wstringop-overflow=] HOT 5
- consider adding as_conll2002
- Vignette coefficients extraction returns NULL HOT 4
- typo
- keep colnames of features in the model object
- add marginal for all labels
- Wstrict-prototypes
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from crfsuite.