yanlesin / sec13flist Goto Github PK

There was a change in rvest 1.0.0 that will make build of package fail.
Workaround is developed.

Only local function could work properly.
Tests need to be adjusted as well

Add github actions for R packages

checking R code for possible problems ... NOTE
SEC_13F_list: no visible binding for global variable 'PDF_STRING'
SEC_13F_list: no visible binding for global variable 'CUSIP_start'
SEC_13F_list: no visible binding for global variable
'ISSUER_NAME_start'
SEC_13F_list: no visible binding for global variable
'ISSUER_DESCRIPTION_start'
SEC_13F_list: no visible binding for global variable 'STATUS_start'
SEC_13F_list: no visible binding for global variable 'CUSIP_end'
SEC_13F_list: no visible binding for global variable 'STATUS_end'
SEC_13F_list: no visible binding for global variable
'HAS_LISTED_OPTION_start'
SEC_13F_list: no visible binding for global variable
'HAS_LISTED_OPTION_end'
SEC_13F_list: no visible binding for global variable 'ISSUER_NAME_end'
SEC_13F_list: no visible binding for global variable
'ISSUER_DESCRIPTION_end'
SEC_13F_list: no visible binding for global variable 'CUSIP'
Undefined global functions or variables:
CUSIP CUSIP_end CUSIP_start HAS_LISTED_OPTION_end
HAS_LISTED_OPTION_start ISSUER_DESCRIPTION_end
ISSUER_DESCRIPTION_start ISSUER_NAME_end ISSUER_NAME_start PDF_STRING
STATUS_end STATUS_start

Travis build failed due to unavailable purrrogress package

functions for SEDOL and ISIN validation

so far 4 tests defined:

1. row count
2. CUSIP checksum number data quality check
3. STATUS field data quality check
4. HAS_LISTED_OPTION field data quality check

sec.gov landing page change and automatic detection of current list url is not working

fill function is the only dependency in the package that requires tidyr. It is possible to include cpp filldown from tidyr as non-exported function, in order to remove dependency.
The remaining dependencies will be pdftools and rvest.
rvest changed in the past and broke functionality and should be considered for removing as dependency.
pdftools - ok to have it as dependency.

use of CUSIP from public sources note

Github action standard check fails test of local function on Windows

In some cases for CINS numbers or CUSIP with character in the middle, function returns incorrectly that identifier is not valid.
Root cause could be that initial CUSIP code was developed with numerical CUSIPs in mind and requires tweaking. Better code could be borrowed from isIsin function, as it handles alpha characters better, but still using the same checksum digit algo

complete test coverage

Need routine to handle download errors if download.file returns status

There is no need to use those packages as dependencies

1. Some CUSIPs are invalid
2. Non-distinct CUSIPs
3. link to FundApps article re CUSIP quality
4. isCusip() function
5. usage example with group_by

handle condition when supplied current year and quarter combo is not available yet.

Replace MAIN_ISSUE field name with HAS_LISTED_OPTION

dplyr 1.1.0 update changed behavior of na_if function causing package to produce an error.

add new functions to readme and docs

2023-Q3 list parsed with errors

improve code coverage for new function

Defaul instead of Default

speed up processing plus eliminate dependency on naniar

The following thing will be fixed in next couple of releases:

1. Description of returned data frame

Tests failed for unfiltered "\f" string in pdf report.
Tests failed when STATUS field parsed between issue and status. This is formatting issue in SEC's pdf header

add option for show/no show progress with purrrogress package.
Default value is off.

pdf_text() accepts URL with PDF. More elegant to read from URL that bring file into local disk

this could be done much simpler than using rvest

create function return correct 9-digit CUSIP from 8-digit CUSIP

SEC13Flist function fail to obtain current year and month causing a stop with error

1. Limit scope of the function to files that could be parsed (starting from Q1 2004)
2. Special handling of file name for Q1 2004
3. Checking parameters passed into function (starting from Q1 2004)

isolate line_separator into function to enable higher test coverage

1. Current list
2. Year and Quarter based lists
3. URL validation

It is hard coded that data starts on page 3 of the file.
It is possible to determine start of data section based on Run Date starting string.

fredr package as an example
remove travis

consider finishing removal of dependencies on dplyr and rlang

Current List section contains most recent 13F list. Use rvest to harvest this and utilize as default value for the function

this will add readr as dependency, but will remove need to use purrr. Also faster.

One security fails parsing due to 1 character longer name.

Remove roxygen template from utils.R

Add news.md

In case download is failing, or webpage structure is changed and page cannot be scrapped properly, offline mode is required for SEC13FList function

Process all parse able PDFs and include them as data into package. Update for new data quarterly

only read_lines used. Consider readLines with text connection

See this for details
http://r-pkgs.had.co.nz/description.html#dependencies

Provide reference to CUSIP checksum algorithm and code used for C++ function.