Comments (12)
Thank you for your contribution @lucmski. I will check the docker files and let you know.
from hoaxy-backend.
Working on Docker (#26). In the meanwhile, an AMI is available for EC2 (see https://github.com/IUNetSci/hoaxy-backend#hoaxy-backend-as-aws-image-ami).
Not sure about other questions...
from hoaxy-backend.
I meant is it possible to distribute a list of twitter usernames to fetch and get their botometer score async like with payloads managed by rabbitmq or celery. Globally, it is more about handling more data and being able to resume tasks. (get the last tweet id scraped so it prevents using resources).
from hoaxy-backend.
Example for a front/server dockerfile (alpine/nodejs v11.x):
https://gist.github.com/lucmski/b4a143b0bf026448f159e6bf463ed0c2
Example for hoaxy-backend (alpine 3.9/mininconda2):
https://gist.github.com/lucmski/87f13b669422c2f057b500404a01d359
If needed, I can do a miniconda3...
from hoaxy-backend.
@lucmski, I'm trying the docker file for hoaxy-backend. I was able to follow first three steps. I'm stucked with step 4 (https://gist.github.com/lucmski/87f13b669422c2f057b500404a01d359#run-docker-container).
I think there is a typo in the first command in that step. (docker -ti --rm lucmichalskip/hoaxy-backend:alpine-miniconda2-jdk8 hoaxy -h)
I'm getting this error. "unknown shorthand flag: 't' in -ti"
from hoaxy-backend.
forgot to add 'run' command sorry, the gist was updated
from hoaxy-backend.
@lucmski Can you share the Dockerfile lucmichalskip/hoaxy-backend:alpine-miniconda2-jdk8 with us. I'm trying to follow your instructions again and got an error in docker run command. I think it will be easier to troubleshoot if I can see the Dockerfile.
chathuri@chathuri-Latitude-E7450:~/IUNI/hoaxy-backend$ docker run -ti --rm lucmichalskip/hoaxy-backend:alpine-miniconda2-jdk8 hoaxy -h
/opt/conda/lib/python2.7/site-packages/hoaxy-0.1.0-py2.7.egg/hoaxy/init.py:30: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
Traceback (most recent call last):
File "/opt/conda/bin/hoaxy", line 11, in
load_entry_point('hoaxy==0.1.0', 'console_scripts', 'hoaxy')()
File "build/bdist.linux-x86_64/egg/hoaxy/commands/cmdline.py", line 80, in main
File "build/bdist.linux-x86_64/egg/hoaxy/utils/init.py", line 20, in list_cls_under_mod
File "/opt/conda/lib/python2.7/importlib/init.py", line 37, in import_module
import(name)
File "build/bdist.linux-x86_64/egg/hoaxy/commands/crawl.py", line 12, in
File "build/bdist.linux-x86_64/egg/hoaxy/crawl/spiders/article.py", line 11, in
File "build/bdist.linux-x86_64/egg/hoaxy/database/init.py", line 27, in
File "/opt/conda/lib/python2.7/site-packages/sqlalchemy/engine/init.py", line 423, in create_engine
return strategy.create(*args, **kwargs)
File "/opt/conda/lib/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 87, in create
dbapi = dialect_cls.dbapi(**dbapi_args)
File "/opt/conda/lib/python2.7/site-packages/sqlalchemy/dialects/postgresql/psycopg2.py", line 605, in dbapi
import psycopg2
File "/opt/conda/lib/python2.7/site-packages/psycopg2/init.py", line 50, in
from psycopg2._psycopg import BINARY, NUMBER, STRING, DATETIME, ROWID
ImportError: libssl.so.1.0.0: cannot open shared object file: No such file or directory
from hoaxy-backend.
Hi,
Sounds that even my dockerfile is outdated; i need to update it.
Why do not you split into separate service with a docker-compose ?
My telegram is @deepocrates for faster communication.
Cheers,
L
from hoaxy-backend.
From fork, https://github.com/lucmski/hoaxy-backend/tree/dockerize-p3/docker
please try:
git clone https://github.com/lucmski/hoaxy-backend
cd hoaxy-backend/docker
docker build -t hoaxy .
docker run -ti --rm hoaxy -h
from hoaxy-backend.
should be interesting you https://github.com/twintproject/twint-search
from hoaxy-backend.
@roscopecoltran, @deepocrates I was able to build docker for hoaxy backend successfully with your instructions. Thank you for the contribution.
But when I run hoaxy -h, it gives me this error.
(base) chathuri@chathuri-Latitude-E7450:~/IUNI/hoaxy/lucmski/hoaxy-backend/docker$ docker run -ti --rm hoaxy -h
/opt/conda/lib/python3.7/site-packages/psycopg2-2.7.6.1-py3.7-linux-x86_64.egg/psycopg2/init.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use "pip install psycopg2-binary" instead. For details see: http://initd.org/psycopg/docs/install.html#binary-install-from-pypi.
""")
**************************** ERROR ******************************************
NO HOAXY CONFIGURATION FILE FOUND: (DEFAULT '/home/.hoaxy/conf.yaml')!
(1) If you have put the conf.yaml to somewhere other than default location
'/root/.hoaxy/',
please set your enviroment variable HOAXY_HOME properly.
(2) Use hoaxy config [--home=] to get a sample of configuration file
named conf.sample.yaml
(3) Edit and rename conf.sample.yaml to conf.yaml. And you should edit and
rename other sample files.
Currently, hoaxy uses default settings and may not work properly.
Usage:
hoaxy [options] [...]
hoaxy -h | --help
hoaxy -v | --version
Subcommands are:
config Generate sample of configuration files
crawl Crawl news sites
init Initialize DB tables and load site data if exists
site News site management: add, alter and dump
lucene Lucene Indexing and Searching
report Report and generate statistics of hoaxy collection
sns Online social network services management
For subcommands, use hoaxy <command> --help
to get the usage of the
subcommand !
Global options are:
--help Show help for this subcommand.
--console-log-level= Set logging level for console output.
[default: debug]
from hoaxy-backend.
@lucmski merged the docker changes to master. Thank you for your contribution.
from hoaxy-backend.
Related Issues (20)
- GET Articles queries only return 100 results (Hoaxy API) HOT 5
- Make API server configurable HOT 1
- Docker build fail on jcc install HOT 1
- Add AltNews.in And BoomLive.in as fact checkers HOT 1
- One-time update of source list HOT 1
- Make it easier and document how to update source list HOT 4
- Docker compose not working HOT 1
- Facebook interface? (and CI) HOT 4
- Some of Hoaxy Web APIs need switch to HTTP POST HOT 1
- Comply with new Twitter API HOT 2
- Fix for Botometer-v4 HOT 3
- A bug in network construction HOT 2
- Hoaxy Community AMI HOT 5
- Error when running Hoaxy config HOT 2
- Error when running hoaxy init HOT 2
- New Error when running hoaxy init HOT 21
- Error building the lucene index HOT 5
- Dockerfile link update HOT 1
- error when building using the dockerfile HOT 2
- docker-compose up Error HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from hoaxy-backend.