ffeast / finam-export Goto Github PK
View Code? Open in Web Editor NEWPython client library to download historical data from finam.ru
License: Apache License 2.0
Python client library to download historical data from finam.ru
License: Apache License 2.0
Looks like finam has changed their metadata cache path.
It used to be https://www.finam.ru/cache/icharts/icharts.js,
but now this link has outdated data (i.e. RIM9 is assigned to "Archived futures" market => it's impossible to download it thru my finam-export tool) and the right link is
https://www.finam.ru/cache/N72Hgd54/icharts/icharts.js
Needs to be fixed somehow
Привет. Мне понравилась ваша библиотека. Ваши примеры отлично работают, но вероятно в силу моего незнания получаю след. результат для получения котировок с рынка акций:
exporter = Exporter()
print('*** ALRS ***')
rub = exporter.lookup(name='ALRS', market=Market.SHARES)
assert len(rub) == 1
INFO:finam.export:Fetching https://www.finam.ru/profile/moex-akcii/gazprom/export/
INFO:finam.export:Fetching https://www.finam.ru/cache/N72Hgd54/icharts/icharts.js
DEBUG:finam.export:Parsing line starting with "var aEmitentIds = [1"
DEBUG:finam.export:Parsing line starting with "var aEmitentNames = "
DEBUG:finam.export:Parsing line starting with "var aEmitentCodes = "
DEBUG:finam.export:Parsing line starting with "var aEmitentMarkets "
Можете подсказать, как будет правильно написать чтобы получить нужную котировку акции?
issue came approximately last month, when i tried to run a script, that work appropriate for 1 year at least.
error came:
Traceback (most recent call last):
File "C:_python_projects\my_finance\stock_quotes.py", line 24, in finam_quote
paper = exporter.lookup(code=ticker, market=market)
File "C:_python_python3.9\lib\site-packages\finam\export.py", line 303, in lookup
return self._meta.lookup(*args, **kwargs)
File "C:_python_python3.9\lib\site-packages\finam\export.py", line 220, in lookup
self._load()
File "C:_python_python3.9\lib\site-packages\finam\export.py", line 167, in _load
meta_url = page.find_meta_file()
File "C:_python_python3.9\lib\site-packages\finam\export.py", line 82, in find_meta_file
html = self._fetcher(self.FINAM_ENTRY_URL)
File "C:_python_python3.9\lib\site-packages\finam\export.py", line 55, in fetch_url
raise FinamDownloadError('Unable to load {}: {}'.format(url, e))
finam.exception.FinamDownloadError: Unable to load https://www.finam.ru/profile/moex-akcii/gazprom/export/: HTTP Error 403: Forbidden
It seems like finam restrict automatic loading, because i can do it from same computer, using web interface
I Tried to fetch and download (csv) 1hour forex data, but it doesn't work.
can somebody help what I'm doing wrong?
python finam-download.py --market CURRENCIES_WORLD --timeframe HOURLY --destdir C:\Users
using the finam-download.py form scripts.
Thanks
Is it just logging imperfection or second day of each month/year/any dataset is always missing in the resulting dataset? @ffeast
Ошибка в теме. Думаю, связано с версией питона 3.10. Появляется при попытке исполнения любого скрипта из примеров.
When you try to load data for EURUSD with 15min period for period more than 1 year.
MAX_DAYS_PER_TIMEFRAME doesn't help.
from finam.export import Exporter, Market, LookupComparator, Timeframe
exporter = Exporter()
ref = exporter.lookup('AAPL', market=Market.USA)
results in error: FinamDownloadError: Unable to load https://www.finam.ru/profile/moex-akcii/gazprom/export/: <urlopen error [Errno -3] Temporary failure in name resolution>
Could it be Finam recognizing that the request is not being made by a browser?
Finam has restrictions on the longest data period that can be requested.
This period seems to depend on the timeframe used.
It's reasonable and prevents users from fetching too much data in one go - i.e. requesting 10-years long tick data would result in enormous data volume.
Now if the user specifies a time period which is too long an error is produced (FinamTooLongTimeframeError
)
It would be nice to have an option resulting in automatic splitting of such long periods so that the user doesn't need to guess whether the period he needs is too long for finam servers.
Of course --delay
argument of the downloader scripts must be honored or even enforced
Hey sorry for commenting on the previous issue, I am new to GitHub.
I am wondering if you can add a code that can get the minute data of Nasdaq or any stock under the USA (BATS) section of finam.
I have tried to write a code to access it, but you seem to be a professional at coding and I could really use your help.
Thanks,
Olaf
For example let's take a look on VTBR M5 data for first hour of 2016.01.04 day:
library | site UI |
---|---|
0.07833999999999999 | 0.0783400 |
0.07841000000000001 | 0.0784100 |
It looks like it's not, but I'd like to make sure. Is there any way to get adjusted data?
When I try to use this library, the following error appears:
from finam import Exporter, Market, LookupComparator
exporter = Exporter()
ticker = 'USDRUB_TOD'
rub = exporter.lookup(name = ticker, market = Market.CURRENCIES)
data = exporter.download(rub.index[0],
market = Market.CURRENCIES,
timeframe=Timeframe.MINUTES1,
start_date=datetime.date(2020, 1, 31),
end_date=datetime.date(2020, 2, 1))
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
C:\Users\8523~1\AppData\Local\Temp/ipykernel_67512/432054812.py in <module>
11 data = exporter.download(rub.index[0],
12 market = Market.CURRENCIES,
---> 13 timeframe=Timeframe.MINUTES1,
14 start_date=datetime.date(2020, 1, 31),
15 end_date=datetime.date(2020, 2, 1))
NameError: name 'Timeframe' is not defined
I see there're ~15 stargazers for this tool.
Guys, is there anything missing in the library what's desperately needed so that it's worth spending spare time on it?
Feel free to comment in Russian if there are troubles writing in English
Thanks
I looked market number in html source - number 520 for crypto.
I added it in const.py file below other markets.
But always the same error - ivalid value for --market. I even tried to delete or comment all other markets in const.py just to see what will happen - and program still has the same error.
Seems to be, that text of the error and array of markets doesn't go from const.py file, but somewhere from finam.ru.
What should i change in project files and what i should type in command prompt in order to add crypto quotes market and be able to download btc\eth ???
tried all queries in txt - nothing work.
btc-quotes-query.txt
When downloading an array of datasets of one contract and the same timeframe, separated, for example, by months in one folder - they will be replacing each other in one file with "contract code - timeframe" filename mask.
The only way is to download no more than one query in one folder by one time, and then rename resulting filename before next query with same contract code and timeframe in this folder. Or prepare different folders and set each query to "their own" folder from the beginning, which might be a little bit messy in the end... With years it may be ok, but with months or shorter - so many folder are needed..
Could filename mask be like "SBRF-TICKS-2020-01-01-2020-01-31" or "SBRF-TICKS-2020-01-01-2020-12-31" ?
It will automatically solve auto-replacing and losing data therefore, while downloading files in one folder by one commands array in cmd. Also it's much more informative than just "SBRF-TICKS" for all possible periods...
Ps. Thanks so much for brilliant lib ;> @ffeast
Добрый день!
Все прекрасно работало, но вот уже несколько дней при запросе котировок выдает ошибку:
File "C:\Users...\finam-export\export.py", line 118, in _parse_js_assignment
raise FinamDownloadError('Unable to parse line: {}'.format(line))
exception.FinamDownloadError: Unable to parse line: п»ї<htmlxmlns="http://www.w3.org/1999/xhtml">
У всех так, Финам что-то поменял?
E.g. ticker SIG.
>>./scripts/finam-lookup.py --contract=SIG
INFO:finam.export:Fetching https://www.finam.ru/profile/moex-akcii/gazprom/export/
INFO:finam.export:Fetching https://www.finam.ru/cache/N72Hgd54/icharts/icharts.js
DEBUG:finam.export:Parsing line starting with "var aEmitentIds = [4"
DEBUG:finam.export:Parsing line starting with "var aEmitentNames = "
DEBUG:finam.export:Parsing line starting with "var aEmitentCodes = "
DEBUG:finam.export:Parsing line starting with "var aEmitentMarkets "
name code market
id
874943 Signet Jewelers Limited SIG 25
481495 Signet Jewelers Limited SIG 517
>>./scripts/finam-download.py --market=SPB --destdir=. --contracts=SIG --timeframe=TICKS --startdate=2020-03-27 --enddate=2020-03-27
INFO:root:Handling SIG
INFO:finam.export:Fetching https://www.finam.ru/profile/moex-akcii/gazprom/export/
INFO:finam.export:Fetching https://www.finam.ru/cache/N72Hgd54/icharts/icharts.js
DEBUG:finam.export:Parsing line starting with "var aEmitentIds = [4"
DEBUG:finam.export:Parsing line starting with "var aEmitentNames = "
DEBUG:finam.export:Parsing line starting with "var aEmitentCodes = "
DEBUG:finam.export:Parsing line starting with "var aEmitentMarkets "
INFO:__main__:Downloading contract id 874943
name Signet Jewelers Limited
code SIG
market 25
Name: 0, dtype: object
INFO:finam.export:Fetching http://export.finam.ru/table.csv?sep=3&at=1&e=.csv&d=d&f=table&dtf=1&MSOR=0&tmf=3&mstimever=1&mstime=on&sep2=1&em=874943&code=SIG&cn=SIG&df=27&yf=2020&dt=27&datf=6&yt=2020&market=25&mf=2&mt=2&p=1
ERROR:__main__:Returned data doesnt seem like a valid csv dataset:
The download URL is for market=25, not for market=517 (SPB)
Ошибка аналогична
##10
Преестали работать примеры по получению котировок.
На странице https://www.finam.ru/profile/mosbirzha-valyutnyj-rynok/eurrubtom-eur-rub/export/?market=41 при выгрузке появилось поле token.
В скрипте страницы:
siteKey3: '6LcUHawUAAAAAKQF_5dMIorMoK57KgVQ444Q8IWN',
getToken: function (callback, action) {
grecaptcha.execute('6LcUHawUAAAAAKQF_5dMIorMoK444gVQQVBQ8IWN', { action: action || 'homepage' }).then(function (token) {
callback(token);
});
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.