GithubHelp home page GithubHelp logo

alpacahq / pipeline-live Goto Github PK

View Code? Open in Web Editor NEW
202.0 46.0 56.0 221 KB

Pipeline Extension for Live Trading

Home Page: https://pypi.org/project/pipeline-live/

License: Apache License 2.0

Python 100.00%
trading trading-bot algotrading algo-trading finance financial-analysis financial-data-analysis zipline pipeline python

pipeline-live's Introduction

⚠️ This software is very outdated and is not recommended for use in new setups ⚠️

We will keep this project up and with the documentation for the parts that remain for those users who are still using this, but we suggest migrating to another tool until we perform a major refactoring of this project to bring it up to speed. If this is something you'd like to see please reach out to us to let us know!

The docs below still talk about zipline and Quantopian however Quantopian has since gone out of business and the below hasn't yet been updated to reflect this.

If you're looking for another modern alternative to zipline, we recommend checking out zipline-reloaded in the meantime while we review committing to a major refactor of this project.

Zipline Pipeline Extension for Live Trading

pipeline-live is an extension for zipline pipeline independently usable for live trading, outside of zipline. While zipline is a great backtesting library, the default Pipeline API requires complicated setup for data bundle, which is often challenging to average users. Quantopian's proprietary data sources such as Morningstar is also not available to many. This library is to address this issue by using online API data sources and simplify the interface for live trading usage. The interface complies the original zipline/pipeline for the most part. For more details about the Pipeline API, please see Quantopian's tutorial and zipline document.

If you are looking to use this library for your Quantopian algorithm, check out the migration document.

Data Sources

This library predominantly relies on the Alpaca Data API for daily price data. IEX Cloud data is also supported, though if too much data is requested, it stops being free. (See the note in the IEX section below.)

Install

pipeline-live is a PyPI module and you can install it using pip command.

$ pip install pipeline-live

This module is tested and expected to work with python 3.6 and later

Example

Here is a simple pipeline example. Please make sure to first set your API keys to these environment variables:

using python

import os
os.environ["APCA_API_KEY_ID"] = <ALPACA_API_KEY>
os.environ["APCA_API_SECRET_KEY"] = <ALPACA_SECRET_KEY>
# if you use the paper endpoint:
os.environ["APCA_API_BASE_URL"] = "https://paper-api.alpaca.markets"

(or do it with bash if you prefer)

from pipeline_live.engine import LivePipelineEngine
from pipeline_live.data.sources.alpaca import list_symbols
from pipeline_live.data.alpaca.pricing import USEquityPricing
from pipeline_live.data.alpaca.factors import AverageDollarVolume
from zipline.pipeline import Pipeline

eng = LivePipelineEngine(list_symbols)
top5 = AverageDollarVolume(window_length=20).top(5)
pipe = Pipeline({
    'close': USEquityPricing.close.latest,
}, screen=top5)

df = eng.run_pipeline(pipe)

'''
        close
AAPL   215.49
AMZN  1902.90
FB     172.90
QQQ    180.80
SPY    285.79
'''

Data Cache

Since most of the data does not change during the day, the data access layer caches the dataset on disk. In case you need to purge the cache, the cache data is located in $ZIPLINE_ROOT/data/daily_cache.

Pipeline API

pipeline_live.engine.LivePipelineEngine

This class provides the similar interface to zipline.pipeline.engine.SimplePipelineEngine. The main difference is its run_pipeline does not require the start and end dates as parameters, and returns a DataFrame with the data for the current date (US/Eastern time). Its constructor accepts list_symbol function that is supposed to return the full set of symbols as a string list, which is used as the maximum universe inside the engine.

Alpaca Data API

The Alpaca Data API is currently the least-limited source of pricing data supported by pipeline-live. In order to use the Alpaca Data API, you'll need to register for an Alpaca account here and generate API key information with the dashboard. Once you have your keys generated, you need to store them in the following environment variables:

APCA_API_BASE_URL
APCA_API_KEY_ID
APCA_API_SECRET_KEY

pipeline_live.data.alpaca.pricing.USEquityPricing

This class provides the basic price information retrieved from Alpaca Data API.

Where Did the Polygon integration go?

Alpaca used to offer an integration with polygon, however with the removal of that integration from the Alpaca api we removed the polygon integration here as well.

IEX Data Source API

To use IEX-source data, you need to sign up for an IEX Cloud account and save your IEX token as an environment variable called IEX_TOKEN.

IMPORTANT NOTE: IEX data is now limited for free accounts. In order to avoid using more messages than you are allotted each month, please be sure that you are not using IEX-sourced factors too frequently or on too many securities. For more information about how many messages each method will cost, please refer to this part of the IEX Cloud documentation.

pipeline_live.data.iex.pricing.USEquityPricing

This class provides the basic price information retrieved from IEX Chart API.

pipeline_live.data.iex.fundamentals.IEXCompany

This provides the DataSet interface using IEX Company API.

pipeline_live.data.iex.fundamentals.IEXKeyStats

This provides the DataSet interface using IEX Key Stats API.

pipeline_live.data.iex.factors

It is important to note that the original builtin factors from zipline does not work here as is, since some of them rely on zipline's USEquityPricing class. This package provides the same set of zipline's builtin factor classes using pipeline_live.data.iex.pricing.USEquityPricing class. For the complete list of builtin factors, please refer zipline document

pipeline_live.data.iex.classifiers.Sector()

A shortcut for IEXCompany.sector.latest

pipeline_live.data.iex.classifiers.Industry()

A shortcut for IEXCompany.industry.latest

pipeline-live's People

Contributors

drew887 avatar jbussdieker avatar sammerry avatar shlomiku avatar trobrock avatar ttt733 avatar umitanuki avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

pipeline-live's Issues

Cannot install -- Mac OS

I have tried installing with Conda, pip, brew, everything -- I get this issue with pip install:
ERROR: Could not find a version that satisfies the requirement websockets>=8.0 (from alpaca-trade-api>=0.29->pipeline-live) (from versions: 1.0, 2.0, 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7, 3.0, 3.1, 3.2, 3.3, 3.4, 4.0.1, 5.0, 5.0.1, 6.0, 7.0)
ERROR: No matching distribution found for websockets>=8.0 (from alpaca-trade-api>=0.29->pipeline-live)

I've also tried installing into a separate environment in Python 3.5 or 2.7, no luck. Please advise!

Replace for IEXCompany.symbol

Hi
Currently only way to filter by symbol name is using IEXCompany.symbol
Is there a way making it available using alpaca dataset or some other source?

If another free solution can be applied to IEXCompany.companyName it would be great 👍

Thanks!

Running an entire zipline algorithm using pipeline_live

Dear Pipeline_live Maintainers,

I am trying to execute a Zipline algorithm using data that comes from pipeline_live (that is, using factors and filters inside pipeline_live.data.iex, for example). In fact, I can create a Pipeline object and select certain stocks according to factors and filters, but I am not able to run the algorithm itself, due to a missing "pipeline_loader" or a missing "adjustment_reader" (which I don't know about).
My question is: is it possible to run an algorithm this way, using data from pipeline_live and the zipline infrastructure? If yes, what should I learn about?

Thanks,
ixplog.

engine.py exception

Traceback (most recent call last):
File "/opt/conda/bin/pylivetrader", line 10, in
sys.exit(main())
File "/opt/conda/lib/python3.6/site-packages/click/core.py", line 764, in call
return self.main(*args, **kwargs)
File "/opt/conda/lib/python3.6/site-packages/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/opt/conda/lib/python3.6/site-packages/click/core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/opt/conda/lib/python3.6/site-packages/click/core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/opt/conda/lib/python3.6/site-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/opt/conda/lib/python3.6/site-packages/click/decorators.py", line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File "/opt/conda/lib/python3.6/site-packages/pylivetrader/main.py", line 161, in run
algorithm.run(retry=ctx.retry)
File "/opt/conda/lib/python3.6/site-packages/pylivetrader/algorithm.py", line 271, in run
return self.executor.run(retry=retry)
File "/opt/conda/lib/python3.6/site-packages/pylivetrader/executor/executor.py", line 119, in run
algo.before_trading_start(self.current_data)
File "/opt/conda/lib/python3.6/site-packages/pylivetrader/algorithm.py", line 244, in before_trading_start
self._before_trading_start(self, data)
File "reversion.py", line 97, in before_trading_start
context.my_output = pipeline_output('my_pipeline')
File "/opt/conda/lib/python3.6/site-packages/pylivetrader/misc/api_context.py", line 62, in wrapped
return getattr(algorithm, f.name)(*args, **kwargs)
File "/opt/conda/lib/python3.6/site-packages/pylivetrader/algorithm.py", line 1077, in pipeline_output
output = eng.run_pipeline(self._pipelines[name])
File "/opt/conda/lib/python3.6/site-packages/pipeline_live/engine.py", line 45, in run_pipeline
tz='utc')
File "pandas/tslib.pyx", line 291, in pandas.tslib.Timestamp.new (pandas/tslib.c:9120)
TypeError: new() takes at least 2 positional arguments (1 given)

Update iexfinance usage for compatibility with v0.4.1

The iexfinance library has deprecated a method which is used here in their latest release. We should use the new version of their method instead.

def get_available_symbols(**kwargs):
    """
    MOVED to iexfinance.refdata.get_symbols
    """
    raise ImmediateDeprecationError("get_available_symbols")

module 'pylivetrader.algorithm' has no attribute 'attach_pipeline'

!pylivetrader run -f domo.py --backend-config config.yaml

Traceback (most recent call last): File "/usr/local/bin/pylivetrader", line 8, in <module> sys.exit(main()) File "/usr/local/lib/python3.6/site-packages/click/core.py", line 829, in __call__ return self.main(*args, **kwargs) File "/usr/local/lib/python3.6/site-packages/click/core.py", line 782, in main rv = self.invoke(ctx) File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke return ctx.invoke(self.callback, **ctx.params) File "/usr/local/lib/python3.6/site-packages/click/core.py", line 610, in invoke return callback(*args, **kwargs) File "/usr/local/lib/python3.6/site-packages/click/decorators.py", line 21, in new_func return f(get_current_context(), *args, **kwargs) File "/usr/local/lib/python3.6/site-packages/pylivetrader/__main__.py", line 273, in run ctx = process_algo_params(ctx, **kwargs) File "/usr/local/lib/python3.6/site-packages/pylivetrader/__main__.py", line 189, in process_algo_params algomodule = get_algomodule_by_path(algofile) File "/usr/local/lib/python3.6/site-packages/pylivetrader/loader.py", line 50, in get_algomodule_by_path return get_algomodule(file, filename) File "/usr/local/lib/python3.6/site-packages/pylivetrader/loader.py", line 63, in get_algomodule exec_(code, ns) File "inout.py", line 20, in <module> from pylivetrader.algorithm import attach_pipeline, pipeline_output ImportError: cannot import name 'attach_pipeline'

[2020-10-22 17:24:29.618828] INFO: Algorithm: livetrader start running with backend = alpaca data-frequency = minute Traceback (most recent call last): File "/usr/local/bin/pylivetrader", line 8, in <module> sys.exit(main()) File "/usr/local/lib/python3.6/site-packages/click/core.py", line 829, in __call__ return self.main(*args, **kwargs) File "/usr/local/lib/python3.6/site-packages/click/core.py", line 782, in main rv = self.invoke(ctx) File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke return ctx.invoke(self.callback, **ctx.params) File "/usr/local/lib/python3.6/site-packages/click/core.py", line 610, in invoke return callback(*args, **kwargs) File "/usr/local/lib/python3.6/site-packages/click/decorators.py", line 21, in new_func return f(get_current_context(), *args, **kwargs) File "/usr/local/lib/python3.6/site-packages/pylivetrader/__main__.py", line 277, in run algorithm.run(retry=ctx.retry) File "/usr/local/lib/python3.6/site-packages/pylivetrader/algorithm.py", line 268, in run self.initialize() File "/usr/local/lib/python3.6/site-packages/pylivetrader/algorithm.py", line 228, in initialize self._initialize(self, *args, **kwargs) File "inout.py", line 41, in initialize algo.attach_pipeline(make_pipeline(context), 'pipeline') AttributeError: module 'pylivetrader.algorithm' has no attribute 'attach_pipeline'

Morningstar support

My Quantopian code uses:

from quantopian.pipeline.data import morningstar
from quantopian.pipeline.filters.morningstar import IsPrimaryShare

What is the equivalent with pipeline-live? Note that I have an Alpaca account.

Cannot install pipeline-live Windows

I just installed pylivetrader using 3 channels (conda-forge, Quantopian, alpacahq) and conda create -n pylivetraderenv pylivetrader. The environment is pretty outdated (Python 3.5.5, Pylivetrader 0.0.20, Websockets 6.0) but it appears to work on the very basic algos. However, I just tried to pip install pipeline-live but it failed to install. The install seemed to go pretty far but the error came right at the end. I'll paste the error below. Any thoughts or help would be greatly appreciated!

sing cached tables-3.6.1.tar.gz (4.6 MB)
ERROR: Command errored out with exit status 1:
command: 'C:\Users\Owner\Anaconda3\envs\pylivetraderenv\python.exe' -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\Users\Owner\AppData\Local\Temp\pip-install-f95bg5wp\tables\setup.py'"'"'; file='"'"'C:\Users\Owner\AppData\Local\Temp\pip-install-f95bg5wp\tables\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' egg_info --egg-base 'C:\Users\Owner\AppData\Local\Temp\pip-pip-egg-info-qz8_dtxw'
cwd: C:\Users\Owner\AppData\Local\Temp\pip-install-f95bg5wp\tables
Complete output (29 lines):
* Using Python 3.5.5 | packaged by conda-forge | (default, Jul 24 2018, 01:52:17) [MSC v.1900 64 bit (AMD64)]
* USE_PKGCONFIG: False
* Found conda env: C:\Users\Owner\Anaconda3\envs\pylivetraderenv
Traceback (most recent call last):
File "C:\Users\Owner\Anaconda3\envs\pylivetraderenv\lib\site-packages\setuptools\msvc.py", line 489, in _find_latest_available_vc_ver
return self.find_available_vc_vers()[-1]
IndexError: list index out of range

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\Owner\AppData\Local\Temp\pip-install-f95bg5wp\tables\setup.py", line 635, in <module>
    libraries=(package.library_name,))
  File "C:\Users\Owner\Anaconda3\envs\pylivetraderenv\lib\distutils\ccompiler.py", line 791, in has_function
    objects = self.compile([fname], include_dirs=include_dirs)
  File "C:\Users\Owner\Anaconda3\envs\pylivetraderenv\lib\distutils\_msvccompiler.py", line 304, in compile
    self.initialize()
  File "C:\Users\Owner\Anaconda3\envs\pylivetraderenv\lib\distutils\_msvccompiler.py", line 197, in initialize
    vc_env = _get_vc_env(plat_spec)
  File "C:\Users\Owner\Anaconda3\envs\pylivetraderenv\lib\site-packages\setuptools\msvc.py", line 185, in msvc14_get_vc_env
    return EnvironmentInfo(plat_spec, vc_min_ver=14.0).return_env()
  File "C:\Users\Owner\Anaconda3\envs\pylivetraderenv\lib\site-packages\setuptools\msvc.py", line 843, in __init__
    self.si = SystemInfo(self.ri, vc_ver)
  File "C:\Users\Owner\Anaconda3\envs\pylivetraderenv\lib\site-packages\setuptools\msvc.py", line 485, in __init__
    self.vc_ver = vc_ver or self._find_latest_available_vc_ver()
  File "C:\Users\Owner\Anaconda3\envs\pylivetraderenv\lib\site-packages\setuptools\msvc.py", line 492, in _find_latest_available_vc_ver
    raise distutils.errors.DistutilsPlatformError(err)
distutils.errors.DistutilsPlatformError: Microsoft Visual C++ 14.0 is required. Get it with "Microsoft Visual C++ Build Tools": https://visualstudio.microsoft.com/downloads/
----------------------------------------

ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

Connection reset by peer

I am using alpaca_trade_api-0.39
I got new issue of connection reset by peer even though nothing is change in my code:

Traceback (most recent call last):
File "lowDollarVolume.py", line 312, in
output = run_pipeline(args.f)
File "lowDollarVolume.py", line 295, in run_pipeline
output = create_pipeline()
File "lowDollarVolume.py", line 109, in create_pipeline
exe_pipeline(eng, "symbolsDB", "step_2", ldv_step_2)
File "/home/idanre1/algo/pipeline/pipeline_wrapper.py", line 22, in exe_pipeline
df = eng.run_pipeline(pipe)
File "/home/idanre1/py3env/lib/python3.6/site-packages/pipeline_live/engine.py", line 77, in run_pipeline
initial_workspace,
File "/home/idanre1/py3env/lib/python3.6/site-packages/pipeline_live/engine.py", line 244, in compute_chunk
to_load, mask_dates, symbols, mask,
File "/home/idanre1/py3env/lib/python3.6/site-packages/pipeline_live/data/alpaca/pricing_loader.py", line 42, in load_adjusted_array
prices = alpaca.get_stockprices(chart_range)
File "/home/idanre1/py3env/lib/python3.6/site-packages/pipeline_live/data/sources/alpaca.py", line 24, in get_stockprices
return get_stockprices_cached(all_symbols)
File "/home/idanre1/py3env/lib/python3.6/site-packages/pipeline_live/data/sources/util.py", line 68, in wrapper
body = func(*args, **kwargs)
File "/home/idanre1/py3env/lib/python3.6/site-packages/pipeline_live/data/sources/alpaca.py", line 22, in get_stockprices_cached
return _get_stockprices(all_symbols, limit, timespan)
File "/home/idanre1/py3env/lib/python3.6/site-packages/pipeline_live/data/sources/alpaca.py", line 43, in _get_stockprices
return parallelize(fetch, splitlen=199)(symbols)
File "/home/idanre1/py3env/lib/python3.6/site-packages/pipeline_live/data/sources/util.py", line 30, in wrapper
task_result = task.result()
File "/usr/lib/python3.6/concurrent/futures/_base.py", line 425, in result
return self.__get_result()
File "/usr/lib/python3.6/concurrent/futures/_base.py", line 384, in __get_result
raise self._exception
File "/usr/lib/python3.6/concurrent/futures/thread.py", line 56, in run
result = self.fn(*self.args, **self.kwargs)
File "/home/idanre1/py3env/lib/python3.6/site-packages/pipeline_live/data/sources/alpaca.py", line 33, in fetch
barset = tradeapi.REST().get_barset(symbols, timespan, limit)
File "/home/idanre1/py3env/lib/python3.6/site-packages/alpaca_trade_api/rest.py", line 342, in get_barset
resp = self.data_get('/bars/{}'.format(timeframe), params)
File "/home/idanre1/py3env/lib/python3.6/site-packages/alpaca_trade_api/rest.py", line 163, in data_get
'GET', path, data, base_url=base_url, api_version='v1'
File "/home/idanre1/py3env/lib/python3.6/site-packages/alpaca_trade_api/rest.py", line 112, in _request
return self._one_request(method, url, opts, retry)
File "/home/idanre1/py3env/lib/python3.6/site-packages/alpaca_trade_api/rest.py", line 131, in _one_request
resp = self._session.request(method, url, **opts)
File "/home/idanre1/py3env/lib/python3.6/site-packages/requests/sessions.py", line 533, in request
resp = self.send(prep, **send_kwargs)
File "/home/idanre1/py3env/lib/python3.6/site-packages/requests/sessions.py", line 686, in send
r.content
File "/home/idanre1/py3env/lib/python3.6/site-packages/requests/models.py", line 828, in content
self._content = b''.join(self.iter_content(CONTENT_CHUNK_SIZE)) or b''
File "/home/idanre1/py3env/lib/python3.6/site-packages/requests/models.py", line 753, in generate
raise ChunkedEncodingError(e)
requests.exceptions.ChunkedEncodingError: ("Connection broken: ConnectionResetError(104, 'Connection reset by peer')", ConnectionResetError(104, 'Connection reset by peer'))

StaticAssets screen

Hi, I'm trying to set a pipeline screen with a list of predetermined stocks using StaticAssets. While this works great in Quantopian, I cannot get it to work as a filter using zipline or pipeline_live. Any help would be great.

Running the example in the README results with an error

Hi
I have running ubuntu 18 LTS
I have installed pipeline-live using the following:
sudo apt install python3 python3-pip python3-tk
pip3 install pipeline-live

then I am trying to run the example in the README of the github which result with the following error
sudo apt install python3 python3-pip python3-tk
python3 pipeline.py
Traceback (most recent call last):
File "pipeline.py", line 1, in
from pipeline_live.engine import LivePipelineEngine
File "/home/idan/.local/lib/python3.6/site-packages/pipeline_live/engine.py", line 11, in
from zipline.lib.adjusted_array import ensure_adjusted_array, ensure_ndarray
File "/home/idan/.local/lib/python3.6/site-packages/zipline/init.py", line 29, in
from .utils.run_algo import run_algorithm
File "/home/idan/.local/lib/python3.6/site-packages/zipline/utils/run_algo.py", line 20, in
from zipline.finance import metrics
File "/home/idan/.local/lib/python3.6/site-packages/zipline/finance/metrics/init.py", line 43, in
from .tracker import MetricsTracker
File "/home/idan/.local/lib/python3.6/site-packages/zipline/finance/metrics/tracker.py", line 19, in
from ..ledger import Ledger
File "/home/idan/.local/lib/python3.6/site-packages/zipline/finance/ledger.py", line 28, in
from zipline.finance.transaction import Transaction
File "/home/idan/.local/lib/python3.6/site-packages/zipline/finance/transaction.py", line 20, in
from zipline.protocol import DATASOURCE_TYPE
File "/home/idan/.local/lib/python3.6/site-packages/zipline/protocol.py", line 20, in
from .utils.enum import enum
File "/home/idan/.local/lib/python3.6/site-packages/zipline/utils/enum.py", line 43, in
method='bfill',
File "/home/idan/.local/lib/python3.6/site-packages/pandas/core/series.py", line 3325, in reindex
return super(Series, self).reindex(index=index, **kwargs)
File "/home/idan/.local/lib/python3.6/site-packages/pandas/core/generic.py", line 3689, in reindex
fill_value, copy).finalize(self)
File "/home/idan/.local/lib/python3.6/site-packages/pandas/core/generic.py", line 3702, in _reindex_axes
tolerance=tolerance, method=method)
File "/home/idan/.local/lib/python3.6/site-packages/pandas/core/indexes/base.py", line 3622, in reindex
tolerance=tolerance)
File "/home/idan/.local/lib/python3.6/site-packages/pandas/core/indexes/base.py", line 3248, in get_indexer
indexer = self._get_fill_indexer(target, method, limit, tolerance)
File "/home/idan/.local/lib/python3.6/site-packages/pandas/core/indexes/base.py", line 3278, in _get_fill_indexer
limit)
File "/home/idan/.local/lib/python3.6/site-packages/pandas/core/indexes/base.py", line 3300, in _get_fill_indexer_searchsorted
side)
File "/home/idan/.local/lib/python3.6/site-packages/pandas/core/indexes/base.py", line 4200, in _searchsorted_monotonic
raise ValueError('index must be monotonic increasing or decreasing')
ValueError: index must be monotonic increasing or decreasing

please help

'AverageDollarVolume' and 'SimpleMovingAverage' missing

For
from pipeline_live.data.alpaca.factors import SimpleMovingAverage, AverageDollarVolume

I get the following errors:

No name 'SimpleMovingAverage' in module 'pipeline_live.data.alpaca.factors'pylint(no-name-in-module)
No name 'AverageDollarVolume' in module 'pipeline_live.data.alpaca.factors'pylint(no-name-in-module)

Polygon Comapny API Common Stock Filter

Pipeline-live is missing fundamental filter for stock type and isOTC. Is there any way to mitigate this, I am looking for something like this:

common_stock = PolygonCompany.type.latest.eq('CS') # for common stock

Could not find a version that satisfies the requirement pandas==0.22.0

Hi, I am trying to install the pipeline-live, but having following error.

  ERROR: Command errored out with exit status 1: 'c:\users\wu\appdata\local\programs\python\python39\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\Wu\\AppData\\Local\\Temp\\pip-install-f_r35rfb\\numpy_67c03d43693c4841981441e89e071a43\\setup.py'"'"'; __file__='"'"'C:\\Users\\Wu\\AppData\\Local\\Temp\\pip-install-f_r35rfb\\numpy_67c03d43693c4841981441e89e071a43\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\Wu\AppData\Local\Temp\pip-record-7t5te5_v\install-record.txt' --single-version-externally-managed --prefix 'C:\Users\Wu\AppData\Local\Temp\pip-build-env-7m8umo9x\overlay' --compile --install-headers 'C:\Users\Wu\AppData\Local\Temp\pip-build-env-7m8umo9x\overlay\Include\numpy' Check the logs for full command output.
  ----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/08/01/803834bc8a4e708aedebb133095a88a4dad9f45bbaf5ad777d2bea543c7e/pandas-0.22.0.tar.gz#sha256=44a94091dd71f05922eec661638ec1a35f26d573c119aa2fad964f10a2880e6c (from https://pypi.org/simple/pandas/). Command errored out with exit status 1: 'c:\users\wu\appdata\local\programs\python\python39\python.exe' 'c:\users\wu\appdata\local\programs\python\python39\lib\site-packages\pip' install --ignore-installed --no-user --prefix 'C:\Users\Wu\AppData\Local\Temp\pip-build-env-7m8umo9x\overlay' --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- wheel setuptools Cython 'numpy==1.9.3; python_version=='"'"'3.5'"'"'' 'numpy==1.12.1; python_version=='"'"'3.6'"'"'' 'numpy==1.13.1; python_version>='"'"'3.7'"'"'' Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement pandas==0.22.0 (from pipeline-live)
ERROR: No matching distribution found for pandas==0.22.0

I am able to do pip install pandas to install the latest, but seems like the library only needs pandas 0.22.0
Any suggestions?

Also is there a way for me to filter all the stocks every 4hours?

Thanks!

Using Mask to speed up execution.

I have a pipeline that has a complex CustomFactor. I am trying to reduce the time it takes to run by using a mask but the mask always has True values. i.e. It doesn't filter out values. The total stocks I start with are ~8000 but if the mask works, my Custom Factor should only execute on about 200. That will increase performance. I am more than happy to implement this if you can point me in the right direction.

Example:

def make_pipeline():
    rsi = RSI()
    custom = CustomFactor(mask=(rsi > 70)) # should ideally provide a subset to run on
    return Pipeline({
        'column': columns
    })

Retry in the case of temporary API error

Sometimes, we see this kind of temporary issue in API and it's better to be able to retry so the entire script doesn't stop.

File "c:\users\john\anaconda3\lib\site-packages\requests\adapters.py", line 508, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='api.iextrading.com', port=443): Max retries exceeded with url: /1.0/stock/market/batch?symbols=STNE%2CSTNG%2CSTNL%2CSTNLU%2CSTNLW%2CSTON%2CSTOR%2CSTOT%2CSTPP%2CSTPZ%2CSTRA%2CSTRL%2CSTRM%2CSTRO%2CSTRS%2CSTRT%2CSTT%2CSTT-C%2CSTT-D%2CSTT-E%2CSTT-G%2CSTWD%2CSTX%2CSTXB%2CSTZ%2CSTZ.B%2CSU%2CSUB%2CSUI%2CSUM%2CSUMR%2CSUN%2CSUNS%2CSUNW%2CSUP%2CSUPN%2CSUPV%2CSURF%2CSUSA%2CSUSB%2CSUSC%2CSVA%2CSVBI%2CSVM%2CSVMK%2CSVRA%2CSVT%2CSVVC%2CSVXY%2CSWAN%2CSWCH%2CSWI%2CSWIR%2CSWJ%2CSWK%2CSWKS%2CSWM%2CSWN%2CSWP%2CSWX%2CSWZ%2CSXC%2CSXCP%2CSXE%2CSXI%2CSXT%2CSYBT%2CSYBX%2CSYE%2CSYF%2CSYG%2CSYK%2CSYKE%2CSYLD%2CSYMC%2CSYN%2CSYNA%2CSYNC%2CSYNH%2CSYNL%2CSYPR%2CSYRS%2CSYV%2CSYX%2CSYY%2CSZC%2CSZK%2CSZNE%2CSZO%2CT%2CTA%2CTAC%2CTACO%2CTACOW%2CTACT%2CTAGS%2CTAHO%2CTAIL%2CTAIT&types=chart&range=2y (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x000002503D0E0390>: Failed to establish a new connection: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond',))

ERROR: pipeline_live.data.iex.classifiers.Sector()

Hi, I am trying to make a pipeline filtering the universe of stocks by industry but I am getting a error that says:

pharma_industry = Industry().search('Pharmaceuticals')
AttributeError: 'Latest' object has no attribute 'search'

What operator can I use so the industry() filters any that includes the string 'pharmaceutical'?

PolygonCompany.country gives error

Code:

from pipeline_live.data.polygon.fundamentals import PolygonCompany
from pipeline_live.engine import LivePipelineEngine
from pipeline_live.data.sources.iex import list_symbols
from zipline.pipeline import Pipeline

--- code for API setup

pipe = Pipeline(
columns={'country': PolygonCompany.country.latest
},
screen=country
)

eng = LivePipelineEngine(list_symbols)

eng.run_pipeline(pipe)

error message:

File "zipline/lib/_factorize.pyx", line 187, in zipline.lib._factorize.factorize_strings
File "zipline/lib/_factorize.pyx", line 222, in zipline.lib._factorize.factorize_strings
File "zipline/lib/_factorize.pyx", line 171, in zipline.lib._factorize.factorize_strings_impl
TypeError: '<' not supported between instances of 'str' and 'NoneType'

Is there a fix?

Best,

IEX Data Causes Symbol Error

Version:
pipeline-live==0.1.7

Using the following example causes an IEXSymbolError. The same error is triggered for a full iex pipeline or mixed iex and alpaca pipeline.

from pipeline_live.engine import LivePipelineEngine
from pipeline_live.data.sources.alpaca import list_symbols
from pipeline_live.data.alpaca.pricing import USEquityPricing
from pipeline_live.data.iex.fundamentals import (IEXKeyStats, IEXCompany)
from pipeline_live.data.alpaca.factors import AverageDollarVolume
from zipline.pipeline import Pipeline

eng = LivePipelineEngine(list_symbols)
top5 = AverageDollarVolume(window_length=20).top(5)
pipe = Pipeline({
    'close': USEquityPricing.close.latest,
    'marketcap': IEXKeyStats.marketcap.latest,
}, screen=top5)

eng.run_pipeline(pipe)
/usr/local/lib/python3.6/site-packages/iexfinance/stocks/base.py in _get_endpoint(self, endpoint, params, fmt_p, fmt_j, filter_)
    108         for symbol in self.symbols:
    109             if symbol not in data:
--> 110                 raise IEXSymbolError(symbol)
    111             if endpoint not in data[symbol]:
    112                 result[symbol] = []

IEXSymbolError: Symbol ARR-A* not found.

Q1500US and Q500US Migration Guide

I am currently migrating my algorithm from Quantopian to Alpaca, and am having a very hard time finding any resources on migrating Q1500US and Q500US Pipeline filters to the Zipline/Pipeline-live API

I asked on the AlpacaHQ Slack channel but didn't get a response.

Thanks!

Is this still working? Could not install Mac OS

I tried everything. Virtual environment with Python 3.6 , 3.9 etc. and could not install innumerable errors.
Is this still working? I did not see any options fixing it here.
Can you advice pls?

---------------------------------------- ERROR: Command errored out with exit status 1: /Users/e/DATA/Environments/project1_env/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/db/j3290d917jsf5tkh5qkfttd80000gn/T/pip-install-bfa0yooq/pandas_8b249392aab84dfd8bc1ba0149d23f4b/setup.py'"'"'; __file__='"'"'/private/var/folders/db/j3290d917jsf5tkh5qkfttd80000gn/T/pip-install-bfa0yooq/pandas_8b249392aab84dfd8bc1ba0149d23f4b/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/db/j3290d917jsf5tkh5qkfttd80000gn/T/pip-record-mjlbq2cs/install-record.txt --single-version-externally-managed --compile --install-headers /Users/el/DATA/Environments/project1_env/include/site/python3.9/pandas Check the logs for full command output.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.