`make io-download-and-unzip

docker-compose up -d
[+] Running 2/2
⠿ Container postgres Running 0.0s
⠿ Container app Started 0.4s

compose-up run app container & [DOWNLOAD]
[+] Running 1/0
⠿ Container postgres Running 0.0s
creating dict files url
ref date will be: '2022-08-15' with 37 out of 37 (100.0%)
creating dict files url
Done
creating /app/src/data/2022-08-15... done!
Traceback (most recent call last):
File "src/io/download.py", line 93, in
main()
File "src/io/download.py", line 35, in main
archive = zipfile.ZipFile(path_save_file, 'r')
File "/usr/local/lib/python3.8/zipfile.py", line 1251, in init
self.fp = io.open(file, filemode)
FileNotFoundError: [Errno 2] No such file or directory: '/app/src/data/2022-08-15/Socios0.zip'
make: *** [Makefile:133: io-download-and-unzip] Error 1
`

Os testes semanais estão falhando por timeout
https://github.com/libercapital/dados_publicos_cnpj_receita_federal/actions/runs/3128047446

Encontrar uma maneira de contornar isso

• add Retry 🤔

Existe uma coluna conforme layout, chamada matriz_filial na tabela estabelecimento.

Quando fiz o import essa coluna não existe na tabela, é de extrema importância.

Valeu!

make db-setup

docker-compose up -d
postgres is up-to-date
Starting app ... done

SETUP
Creating db
Starting postgres ... done
Traceback (most recent call last):
File "", line 1, in
File "/app/src/db_models/utils.py", line 4, in
from src.db_models.models import dict_db_models
File "/app/src/db_models/models.py", line 31, in
class Company(Base, DBModelConfig):
File "/root/.local/lib/python3.8/site-packages/sqlalchemy/orm/decl_api.py", line 72, in init
as_declarative(reg, cls, dict)
File "/root/.local/lib/python3.8/site-packages/sqlalchemy/orm/decl_base.py", line 126, in as_declarative
return MapperConfig.setup_mapping(registry, cls, dict, None, {})
File "/root/.local/lib/python3.8/site-packages/sqlalchemy/orm/decl_base.py", line 177, in setup_mapping
return cfg_cls(registry, cls, dict_, table, mapper_kw)
File "/root/.local/lib/python3.8/site-packages/sqlalchemy/orm/decl_base.py", line 322, in init
self._setup_table(table)
File "/root/.local/lib/python3.8/site-packages/sqlalchemy/orm/decl_base.py", line 811, in _setup_table
table_cls(
File "", line 2, in new
File "/root/.local/lib/python3.8/site-packages/sqlalchemy/util/deprecations.py", line 309, in warned
return fn(*args, **kwargs)
File "/root/.local/lib/python3.8/site-packages/sqlalchemy/sql/schema.py", line 584, in new
raise exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Table '' is already defined for this MetaData instance. Specify 'extend_existing=True' to redefine options and columns on an existing Table object.
make: *** [Makefile:82: db-setup] Error 1

Em uma das últimas etapas, estou obtendo um erro.

Alguém passou por isso?

Vou descrever todo log desde a execução até o erro:

$ make engine-company
'

docker-compose up -d
postgres is up-to-date
Starting app ... done

compose-up run app container & [ENGINE COMPANY]
Creating dados_publicos_cnpj_receita_federal_app_run ... done
creating dict files url
ref date will be: '2024-01-16' with 37 out of 37 (100.0%)
No pk found on: 'rf_company'
Pk not found on: 'rf_company'
No indexes found on: 'rf_company'
Can't delete 'ix_rf_company_cnpj_root' on :'rf_company' --> index does not exists
No indexes found on: 'rf_company'
Can't delete 'ix_rf_company_cnpj' on :'rf_company' --> index does not exists
[ 1/9] Getting file K3241.K03200Y1.D40113.ESTABELE
2024-01-20 20:49:49 | sending to db
ERROR: 137
make: *** [Makefile:147: engine-company] Error 137
root@amplisalev1:/dados_publicos_cnpj_receita_federal# ^C
root@amplisalev1:/dados_publicos_cnpj_receita_federal#
'

O link do recurso:
https://dados.gov.br/dados/conjuntos-dados/cadastro-nacional-da-pessoa-juridica-cnpj

Não existe mais e redireciona para esse link:
https://dados.gov.br/error

Isso acaba quebrando o make io-download-and-unzip

/app/src/io/get_files_dict.py:71: MarkupResemblesLocatorWarning: The input looks more like a filename than markup. You may want to open this file and pass the filehandle into Beautiful Soup.
soup_tax_regime = BeautifulSoup(page_tax_regime.text, 'html.parser')
Traceback (most recent call last):
File "src/io/download.py", line 97, in
main()
File "src/io/download.py", line 17, in main
dict_files_dict = get_files_dict()
File "/app/src/io/get_files_dict.py", line 74, in main
rows_tax_regime = table_tax_regime.find_all('tr')
AttributeError: 'NoneType' object has no attribute 'find_all'
make: *** [io-download-and-unzip] Error 1

Olá,
erro durante a instalação, na etapa de execução do comando: $ make db-setup

Vide imagem:

Parece que a imagem: image: dados_publicos:1.0
não está disponível.

Muito bom, pesquisei exatamente na hora que voce publicou, parabens!!!

Porém faltou você colocar tudo que precisa ser instalado, docker, make....