Comments (3)
Per discussion with Neal, assigning this to myself.
from housing-insights.
@prisalex and I worked on this today. We sketched out the blueprint for the full HISql object and wrote some preliminary code for the DataSql subclass. I committed our work to my forked repo in an add-datawriter
branch. We had a few questions about how we'll be reading from and updating information in the database, but I'll put those through when they're more organized.
I'm going to try to work on the DataSql object some more tomorrow, but may not be able to pick it up again until Saturday.
from housing-insights.
@salomoneb and @prisalex - great! Reminder, I'm out of town for the week starting this afternoon (ski trip with my dad). I'll have my computer and will try to stay on top of Git and Slack messages, but will be slower to respond.
On reading and updating, not sure what your questions were, but here's what I had in mind:
- python/housinginsights/tools/dbtools.py has functions to get SQLAlchemy engine and connections. Feel free to move these into object oriented structure, though might be more flexible to leave them where they are and have the HISql object call them.
- Make sure to include a 'db_choice' parameter for HISql with default of 'housinginsights_local' or somesuch; the dbtools.py has ways to choose between local and remote.
- Obviously, we want to use meta.json for the appropriate field names and variable types to create the table if it doesn't exist yet.
- ManifestSql writes one row at a time. Create the table if it doesn't exist yet.
- DataSql uses COPY FROM to append the whole file to the bottom of the table.
As always, adapt as needed.
from housing-insights.
Related Issues (20)
- Convert welcome modal to Bootstrap or fix semantic ui sizing issue HOT 1
- Refactor `_populate_zone_facts` to be cleaner
- Calculate percent subsidized by zone
- Add lat/lon to geocode from mar cleaner step
- Populate 'in_prescat', 'in_dhcd', 'in_dchousing' fields
- Troubleshoot why 'Most Recent REAC date' and 'most recent TOPA date' sliders are bunched up on load HOT 1
- Dedupe address match failing due to integer MAR IDs HOT 1
- Add all project fields to the filter api endpoint
- User Feedback? HOT 2
- Persist missing address id in create_project_subsidy_csv() when returned from mar api call
- test_load_data.py failing due to refactored _populate_zone_facts HOT 1
- Bug in Range Breakdown Displayed (Zone Type: Census Tract && Data Choices: Crime Rate: Violent 12 months) HOT 1
- Inaccurate residential_units counts for at least three census tracts HOT 2
- DHCD addresses not found in MAR HOT 2
- Data retry now prevents null lat/lon project view pages from loading HOT 2
- Preservation Catalog November 2017 update HOT 1
- Front-end refactor master issue
- Track client-side JavaScript errors Rollbar.js HOT 6
- Back-end refactor and data validation master issue
- Possible Data or Import issue (Chastleton)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from housing-insights.