GithubHelp home page GithubHelp logo

kr900910 / mortgage_data_analysis Goto Github PK

View Code? Open in Web Editor NEW
17.0 2.0 9.0 7.63 MB

ETL process which downloads, transforms, and loads Freddie Mac/Fannie Mae mortgage data

Shell 47.32% Python 52.68%
etl-pipeline mortgage-data-analysis hive tableau python shell-script fannie-mae fraddie-mac

mortgage_data_analysis's Issues

download_fannie_mae.py downloads xml and not the actual file

Hi, I'm also working in building a script to automatically download Freddie and Fannie data, the steps that we are using do not work anymore for Fannie I guess. It just downloads an XML file and nothing else.

I was using urllib.request.urlretrieve to fetch the data.
with requests.Session() as sess: print(" SUCCESS : Program Started Succesfully") sess.get(login_page_url) js_session_cookie = sess.cookies.get_dict() login_payload = { 'username': USERNAME, 'password': PASSWORD, 'cookie': js_session_cookie } response_login = sess.post(login_page_url, data=login_payload) download_page_payload = { 'accept': 'Yes', 'action': 'acceptTandC', 'acceptSubmit': 'Continue', 'cookie': js_session_cookie } response_download = sess.post( download_page_url, data=download_page_payload) print(" SUCCESS : Login into freddiemac.embs.com succesful ") urllib.request.urlretrieve('https://loanperformancedata.fanniemae.com/lppub/publish_aws?file=2018Q1.zip','test.zip')
The above code is in python and not working, neither do your steps, which are almost similar I believe.

So if you can rerun your script and if it is working please let me know. thanks in advance.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.