API is a Go written program used by other process to access data
- use scalable database system
- The API process connect to the database (specified by env variable NATS_URI)
- Start a http API using gorilla/mux and wait for request
REST API used by other processes to gather data
License: GNU General Public License v3.0
Once the persister is refectored to use the API it would be great to change the database technology to something more adapted.
Cassandra DB looks a good choice.
It would be cool to benchmark the results
After #4 is merge the following change should be applied:
when an application/html content type is being retrieved the API must extract and replace all absolute/relative URLs for resources with path to the API.
f.e: when requesting example.org at time 20191091551032 with content
<html>
<img src="/an-image.png">
<a href="https://google.de">
<html>
should be transformed to:
<html>
<img src="_<api_uri>_/content?date=20191091551032&url=example.org/an-image.png">
<a href="_<api_uri>_/content?date=20191091551032&url=https://google.de">
<html>
this will allow complete website reading from trandoshan, including image, stylesheets, etc...
GET /resources
with the following query parameters:
the API will retrieve all content with the matching resource-url and the will pick up the closest one in time (absolute distance will be used).
If no resource is available or if the existing resource is too far in time (configurable threshold)
the API will return nothing
if the search query parameters exists a text search will be performed using the Elasticsearch instance instead.
Currently the Persister process interact directly with the database, I'd like to change that by adding an endpoint in the API process to save content. By doing this refactoring the API will be the only process to interact with the database which would be great if I want to change the underlying database technology.
Things to do:
Things to think:
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.