GithubHelp home page GithubHelp logo

ozbarshalom / israeli-bank-scrapers Goto Github PK

View Code? Open in Web Editor NEW

This project forked from eshaham/israeli-bank-scrapers

0.0 1.0 0.0 856 KB

Provide scrapers for all major Israeli banks and credit card companies

License: MIT License

JavaScript 100.00%

israeli-bank-scrapers's Introduction

Israeli Bank Scrapers - Get closer to your own data

Logo

NPM

npm version dependencies Status devDependencies Status

What's here?

What you can find here is scrapers for all major Israeli banks and credit card companies. That's the plan at least. Currently only the following banks are supported:

Prerequisites

To use this you will need to have Node.js >= 6 installed.

Getting started

To use these scrapers you'll need to install the package from npm:

npm install israeli-bank-scrapers --save

Then you can simply import and use it in your node module:

const { createScraper } = require('israeli-bank-scrapers');

const credentials = {...}; // different for each bank
const options = {...};
const scraper = createScraper(options);
const scrapeResult = await scraper.scrape(credentials);

if (scrapeResult.success) {
  scrapeResult.accounts.forEach((account) => {
    console.log(`found ${account.txns.length} transactions for account number ${account.accountNumber}`);
  });
}
else {
  console.error(`scraping failed for the following reason: ${scrapeResult.errorType}`);
}

The definition of the options object is as follows:

{
  companyId: string, // mandatory; one of 'hapoalim', 'leumi', 'discount', 'otsarHahayal', 'visaCal', 'leumiCard', 'isracard', 'amex'
  startDate: Date, // the date to fetch transactions from (can't be before the minimum allowed time difference for the scraper)
  combineInstallments: boolean, // if set to true, all installment transactions will be combine into the first one
  showBrowser: boolean, // shows the browser while scraping, good for debugging (default false)
  verbose: boolean, // include more debug info about in the output
  browser : Browser // optional option from init puppeteer browser instance outside the libary scope. you can get browser diretly from puppeteer via `puppeteer.launch()` command. 
}

The structure of the result object is as follows:

{
  success: boolean,
  accounts: [{
    accountNumber: string,
    txns: [{
      type: string, // can be either 'normal' or 'installments'
      identifier: int, // only if exists
      date: string, // ISO date string
      processedDate: string, // ISO date string
      originalAmount: double,
      originalCurrency: string,
      chargedAmount: double,
      description: string,
      memo: string, // can be null or empty
      installments: {
        number: int, // the current installment number
        total: int, // the total number of installments
      },
      status: string //can either be 'completed' or 'pending'
    }],
  }],
  errorType: "invalidPassword"|"changePassword"|"timeout"|"generic", // only on success=false
  errorMessage: string, // only on success=false
}

You can also use the SCRAPERS list to get scraper metadata:

const { SCRAPERS } = require('israeli-bank-scrapers');

The return value is a list of scraper metadata:

{
  <companyId>: {
    name: string, // the name of the scraper
    loginFields: [ // a list of login field required by this scraper
      '<some field>' // the name of the field
    ]
  }
}

Specific definitions per scraper

Bank Hapoalim scraper

This scraper expects the following credentials object:

const credentials = {
  userCode: <user identification code>,
  password: <user password>
};

This scraper supports fetching transaction from up to one year.

Bank Leumi scraper

This scraper expects the following credentials object:

const credentials = {
  username: <user name>,
  password: <user password>
};

This scraper supports fetching transaction from up to one year.

Discount scraper

This scraper expects the following credentials object:

const credentials = {
  id: <user identification number>,
  password: <user password>,
  num: <user identificaiton code>
};

This scraper supports fetching transaction from up to one year (minus 1 day).

Known Limitations

  • Missing memo field

Bank Otsar Hahayal scraper

This scraper expects the following credentials object:

const credentials = {
  username: <user name>,
  password: <user password>
};

This scraper supports fetching transaction from up to one year.

Visa Cal scraper

This scraper expects the following credentials object:

const credentials = {
  username: <user name>,
  password: <user password>
};

This scraper supports fetching transaction from up to one year.

Leumi-Card scraper

This scraper expects the following credentials object:

const credentials = {
  username: <user name>,
  password: <user password>
};

This scraper supports fetching transaction from up to one year.

Isracard scraper

This scraper expects the following credentials object:

const credentials = {
  id: <user identification number>,
  card6Digits: <6 last digits of card>
  password: <user password>
};

This scraper supports fetching transaction from up to one year.

Amex scraper

This scraper expects the following credentials object:

const credentials = {
  id: <user identification number>,
  card6Digits: <6 last digits of card>
  password: <user password>
};

This scraper supports fetching transaction from up to one year.

Known projects

These are the projects known to be using this module:

Built something interesting you want to share here? Let me know.

License

The MIT License

israeli-bank-scrapers's People

Contributors

eshaham avatar eransakal avatar matanelgabsi avatar dependabot-support avatar chilikla avatar asfktz avatar ifeins avatar sebikaplun avatar brafdlog avatar sagi-mh avatar lielran avatar erezd avatar erezsavir avatar mellowcandle avatar ptzafrir avatar dependabot-preview[bot] avatar

Watchers

James Cloos avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.