stlim0730 / farmview Goto Github PK

As the most important file in this Django project, settings.py must be in better shape in four senses.

1. Scrub sensitive information from the file. Only the template is shared on the remote repository. Each contributor fill out the template on the local workspace.
2. Allow flexible settings on database, web server, etc. It's currently hard-coded for Heroku-specific settings.
3. Remove duplicate declarations and assignments. It has plenty of them now.
4. Include missing details such as admin information. It's not crucial to run the application, but it should have as much information as possible.

This issue is also a starting point of #37. I'm assigning myself.

So I have pulled the recent version, and when I try to run to fix/repro issues with my code, I keep getting error: No modules name comments. Wasn't sure where this is used or the error just appears to me.

I have google searched and included that 'django.contrib.comments' in INSTALLED_APPS in settings.py, did pip install but no luck.

If it's just me getting the error, how should I resolve this?

Since Farmview's official Django version is 1.9, the deprecated settings should be configured. The Django server shows this warning when it starts.

?: (1_8.W001) The standalone TEMPLATE_* settings were deprecated in Django 1.8 and the TEMPLATES dictionary takes precedence. You must put the values of the following settings into your default TEMPLATES dict: TEMPLATE_CONTEXT_PROCESSORS.

As a warning not an error, this can be done after the workshop. It's still good to start taking a look relevant documentation, though.

This is most likely because Carto was updated. Recheck privacy settings and publishing settings and see if mapbook reflects changes

Even though we need to find a good way to smoothly join the three data tables on-the-fly, the basic query features should work.

This issue includes debugging query feature for survey data (currently broken) and finding a reasonable way to present query results without join operations.

I'm assigning myself to this issue.

The blog feature is integrated , but we need images to go to a database. Also fix bugs in the commenting feature

1. On July 30, I'm not seeing detailed info for survey data. If I try and review detail for data_point I just see the parcel data. I do see join survey and parcel data on the preview thumbnail popup

2. If you view the detail popup, then close it and then try and open it again, detail popup is blank. Closing the thumbnail pop-up and re-clicking resolves the issue

Is farmview styling these data fields or is carto?

I assume this is because we are updating the fields and code ... But just making sure.

@stlim0730 Everything else on farmview is working however. renders fine on my mapbook

/map doesn't render anything after making changes on Carto.
@calo1, let's talk on issues, not PRs. Did you try http? What changes did you make?

One of several shameful aspects of this project is that the development environment shares resources with the production environment. Not all of it, but one of the most critical parts, database. The two environments need to get separated.

This assignment includes:

• Adjust Django settings.
• Separate credentials used in this project. It includes 3rd party services such as Dropbox, CartoDB, etc. Also, find a good way to share & manage them within the team.
• Set up vagrant (https://www.vagrantup.com/) for this project. The virtual machine should have all the capabilities required.
• Update README.MD accordingly.

Waiting for a volunteer.

We successfully used the import API to upload new data. But when we import a second time, CartoDB doesn't replace the table it makes a new table with an arbitrary name.

Therefore, syncing is the desired outcome

I've been thinking this project has to more smartly maintain query fields and workflow to manage them for a long time. Adam and I commonly had a good excuse, which the MVP doesn't require all the features we need, to defer the tasks. So, I hard-coded most of the features related to query fields to have simple and limited functionalities.

According to the latest updates on query field requirement document (https://docs.google.com/document/d/1bK6pKhQRQbSd9EzEprI12aMkCbYrvqmiwgYHuaNMz38/edit), the number and the complexity of the query fields the tool should handle have both increased. Now I should say it's the time for the deferred task; query fields should be model-based objects that are better structured and easily configurable to support conditional display, rather than hard-coded.

Also, the model will be renamed into datafield to generally used by query and display (view details) features.

I just finished modeling of query field object. I'll put the data based on the requirement doc and work on the conditional display.

The absence and/or failure to update of css on the local server is preventing some testing and development.

there are two incidents of this happening:

1. about page changes are not updated
2. admin page no css in sources.

KG is "guessing its any page"

When I run the server app on my localhost and visit admin panel, it looks great in Karin's design.

However, when I visit the same page running on the production server, the admin page shows Django's default design, which is %@$%#^.

Any thoughts?

Investigating a parcel on the map application. If one views the detail view, the extended details are displayed. If you close this window and then attempt to reopen it with the "view details about this parcel button" it will not show any data. IF you close the original small pop-up and then attempt to view details, the details are displayed.

Bofan is gone! We need his help. We have a great project for him. If only he would work on this experiential opportunity he would really learn and get great credentials.

How views.py serializes and transfers choices data seriously affects the behavior of the application: if it serializes data into json string, on-the-fly conversion from data to human-readable texts in details window works fine, but query UIs (select_one and select_multiple types) get broken. If the views.py serves just Python dictionary without serialization, query UIs work, but human-readable values are gone.

Currently, it serves raw data to make query UI alive.

Not perfect but easy solution: views.py may serve both serialized and raw version of choices data for corresponding purposes.

Perfect solution: learn why raw data makes trouble in the conversion process.

I'm assigning this task to myself.

I suppose that when data_polygon_temp is removed, data_polygon's CSS has gone together. Data is handed from Carto to Farmview, but it doesn't appear on the map.

@calo1, could you check on Carto?

What's missing in the datafield documentation (https://docs.google.com/document/d/1bK6pKhQRQbSd9EzEprI12aMkCbYrvqmiwgYHuaNMz38/edit) is the type of each datafield. I made two blank columns in the tables: Type and Choices.

• Type column will have one of the three possible query UI types the application currently support; Text, Range, or Select One
• Choices column will have exact string values that CartoDB dataset has (e.g., welloffsite, part_of_farm_available). Each value is spearated by a new line.
• The only type that requires choices column to be filled is Select One.

When I worked on datafield, I used to go to CartoDB to see what type a column has. As the project needs to handle more objects and higher complexity of the model, the type definition should go into the table.

Easily fixable, I think.

Being on the same page is the most important thing in collaboration. This issue is about making our local repos be on the same page from now on.

1. If you have any local changes (compared to current master), make copies of the files you've changed in a separate location. If you don't have any, skip this step.
2. Remove your local repo.
3. Clone the remote repo. This will include all the latest updates.

This is it for the local repo. Now for you:

1. Don't use git add . or git add *.*.

   • When you add files that have been tracked (or existing files in the repo), use git add -u. It's simple enough to add all the files with just ten letters of command. No wild card characters at all.

     -u
     --update
     Update the index just where it already has an entry matching . This removes as well as modifies index entries to match the working tree, but adds no new files.

     If no is given when -u option is used, all tracked files in the entire working tree are updated (old versions of Git used to limit the update to the current directory and its subdirectories).
     https://git-scm.com/docs/git-add

   • When you add files you just created (or new files), use your extreme discernment rather than git add *.*.

     • You can always specify multiple files that really need to be added using wild card characters. For example, git add *.py is much more specific than git add ..
2. Don't add files (especially, binaries) that are not necessary.
   • One of the biggest reason I encouraged you to clone the repo again is to share .gitignore, which specifies what files we avoid to add into the repo (I recommend you take a look at the file).
   • Every single file in the repo must have a good reason to be added.
3. If two or more people are remotely working on the same branch, pay attention to what files they are working on.
   • You can do this on Github by navigating commits on the branch.
   • Another way of doing this is to browse the history of the file of your interest on Github. Make sure you're on the right branch.
4. Commit often, with good names.
   • Obviously.
5. Conflict first
   • We haven't experienced serious merge conflicts yet, which is great. If we have one, we'll stop everything until we resolve it.
6. Django-specific rule: .py files in migrations are very important.
   • They have database manipulation codes that automatizes model changes. If you lose them, you lose much of database.

That's it. Now we're ready to be on the same page.

I think queries that users generate (especially in the workshop) might be a valuable datasource that informs what the users are interested in. For later analysis, we need to save the queries, if possible.

This also reminds me of short conversation with @calo1 about usage-analysis. Google analytics might be a good addition at this moment.

So, this issue includes two-fold analytics methods: a SQL logging feature and Google Analytics integration.

To better organize and serve static files for the two apps (pages and map), this project will have one source (or directory path) of static files. We'll work on the same branch refactoring.

farmview/new_settings.py on this branch is a template of the settings we will use in the near future (this file will simply replace farmview/settings.py). The Django doesn't actually access it for now; It's rather documentation. You can test whether everything is working correctly by temporarily replacing it during the development.

• BASE_DIR will have /static directory for both apps. They currently have their own static directories. We want to put all the static file resources in one place.
  • Group by resource type (css, js, etc.)
  • 3rd party resources like Cartodb.js, Bootstrap, etc. go into their own subdirectory.
  • 3rd party resources like lightbox2, which come out as packages, go into their own subdirectory
  • farmview/new_settings.py has the directory settings we want to use.
  • The structure inside /static will look similar to this:

/
/static
    /css
        /bootstrap
        style.css (where our custom styles go)
    /font
    /js
        /bootstrap
        /cartodb
        /jquery
    /img
    /lightbox2 (where packaged 3rd party resources go)

• In an HTML code, /static/img/foo.jpg will automatically generate the correct URL reference. This is what farmiview/settings.py does for you. You won't need static template tags, such as {% load static from staticfiles %}.
  • During the development, you may want to check if Django can find the file you want using the built-in command line tool below.

$ python manage.py findstatic --verbosity 2 /static/foo/bar.jpg

• The two apps, map and pages, should share css for consistent HTML styling. They now have similar but different copies of it. This should be manually resolved, but it won't be hard.
  • Modularized css might sound more fascinating, but map / pages is not a good distinction for modules.
• Unused static files should be removed. We have many.

As you might have noticed, this issue requires frequent code-push and communication. One reason is that we're working on the same branch. Keep pushing your code updates with brief explain texts. For example,

I moved folder A to B. To make C work with B relocated, I changed D in file E.

When you saw any push on this branch, you should pull it (git pull on the branch refactoring) to be on the same page.

@calo1, Mapbook has some maps with zooming feature using scroll wheel enabled. All the maps we have edit permission, or authored by Farmview account, should disable this feature.

I had a user test the webpage and the first thought was that at the home page they wanted to interact with the main map image but couldn't. This image ifs fine, but is should be a link that directs to the "Map" tab.

While working on #74, I learned that the front end

1. doesn't have static files well-organized
2. doesn't load them in the way Django template supports

The static files should be better hosted and accessed. I self-assign, but also include others who worked on the front end side for reference.

Getting these errors when pressing add blog entries page on live version and production server, respectively.

When you click "go to map" on the landing page.. you end up in an unrecognized link. map/ is running, but the button links incorrectly.

Map url: http://farmview.herokuapp.com/map/

Go to map url: http://farmview.herokuapp.com/map.html

We have lots of unnecessary -- also, not recommended to be distributed -- files in the repository. Actually, it's my fault that I didn't first share repo maintenance guideline with you, due to my trip and busy days at the beginning of this summer.

Also, issue #37 can't begin until this issue is resolved.

@sohyeonlee, @kbourque, @nutelIa, and @calo1, I may ask questions through this issue thread to learn what file is needed for what. I'm assigning myself for this issue.

@calo1, if the only purpose of the polygonaspoints layer is to define the data_polygon layer's behavior in case of specific zoom levels, you can copy css of polygonaspoints and append it to css of data_polygon. Please drop polygonaspoints layer after that. By doing this, data_polygon will still have the same behavior of polygonaspoints for the specific zoom levels. Trust me.

I'm so happy that this layer will be gone. Yay x 10,000. I've hated it, but it took so long until I understand why you wanted it and how it has served the purpose.

If the layer has other reasons to exist, let me know before you drop it.

Some survey data has audio recordings by the respondents. It'll be good to have audio playback feature for it.

One expected issue is the file type. As the respondents use different platforms to submit the survey, recordings might have different formats. Some formats might not be even supported by modern web browsers.

I'm assigning this task to myself.

error was : could not find referrer header.

I don't remember when it started, but recently I see the app failing to load when connected via https protocol. I think there's something to do with http resources from CDN. Investigation needed.

I was trying to test some on-map interactivity features on Safari, to make sure nothing is Chrome-dependent. Safari failed to properly load the map.

Need to find out the reason and solution.

Natalia, you said you would look at the html in the contact us page and request acceptable photos and descriptions from the summer's team. Can you do this?

ProgrammingError at /admin/map/formdata/
relation "map_formdata" does not exist
LINE 1: SELECT COUNT() AS "__count" FROM "map_formdata"
^
Request Method: GET
Request URL: http://127.0.0.1:8000/admin/map/formdata/
Django Version: 1.9.7
Exception Type: ProgrammingError
Exception Value:
relation "map_formdata" does not exist
LINE 1: SELECT COUNT() AS "__count" FROM "map_formdata"
^
Exception Location: /Users/irenelee/Desktop/projects/farmview/vir/lib/python2.7/site-packages/django/db/backends/utils.py in execute, line 64
Python Executable: /Users/irenelee/Desktop/projects/farmview/vir/bin/python
Python Version: 2.7.10
Python Path:
['/Users/irenelee/Desktop/projects/farmview',
'/Users/irenelee/Desktop/projects/farmview/vir/lib/python27.zip',
'/Users/irenelee/Desktop/projects/farmview/vir/lib/python2.7',
'/Users/irenelee/Desktop/projects/farmview/vir/lib/python2.7/plat-darwin',
'/Users/irenelee/Desktop/projects/farmview/vir/lib/python2.7/plat-mac',
'/Users/irenelee/Desktop/projects/farmview/vir/lib/python2.7/plat-mac/lib-scriptpackages',
'/Users/irenelee/Desktop/projects/farmview/vir/Extras/lib/python',
'/Users/irenelee/Desktop/projects/farmview/vir/lib/python2.7/lib-tk',
'/Users/irenelee/Desktop/projects/farmview/vir/lib/python2.7/lib-old',
'/Users/irenelee/Desktop/projects/farmview/vir/lib/python2.7/lib-dynload',
'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7',
'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin',
'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk',
'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac',
'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages',
'/Users/irenelee/Desktop/projects/farmview/vir/lib/python2.7/site-packages']
Server time: Mon, 18 Jul 2016 16:51:09 -0700

I get this error when I try populating data into new Formdata model. The table is not created and not sure how to.

One issue is that the web browser will block non-CartoDB iframes if they're served over HTTP, but not HTTPS. Roughly speaking, a secure webpage served over HTTPS (like Farmview) cannot contain elements served over HTTP where the web browser says the mixed content issue occurred. If you want to add non-CartoDB iframe hosted somewhere else on the Mapbook, make sure the host serves it over HTTPS and you add it on the admin panel with https:// url (e.g., <iframe src="https://some.server.com/..." ...).

Also, I found that mapbook.html had a link to a Google Font resources served over HTTP. An easy walkaround is that you omit the protocol part of the whole url (e.g., change http://fonts.google.com/... into //fonts.google.com/...).

Adam,

I deployed my local branch to the production server for test. https://farmview.herokuapp.com/
On the admin panel (https://farmview.herokuapp.com/admin), you can add/delete/edit Mapbook items. I put one entry as an example.

If you confirm this feature, I'll merge this branch into master and redeploy master on the server. Questions are welcomed!

Right now, map interaction is working so I don't want to change anything ...

But the data_polygon layer used to have a sql query like this
SELECT ST_Centroid(points.the_geom_webmercator) , point.* FROM calo1.data_polygon as point

to create a layer called point at the centroid of the polygons.

Then part of the CSS looks like this:

#point{
   marker-fill-opacity: 0.9;
   marker-line-color: #FFFFFF;
   marker-line-width: 3.5;
   marker-line-opacity: 1;
   marker-placement: point;
   marker-type: ellipse;
   marker-width: 15.5;
   marker-allow-overlap: true;
  [zoom > 12]{marker-fill-opacity: 0;
  marker-line-width: 1}

}


#point{[zoom <=12][survey_session_desc="exisiting_ranch"]{ 
   marker-fill: #58faf4;
    }}
#point{[zoom <=12][survey_session_desc="site_visit"]{ 
   marker-fill: #ff0040;
    }}

This tells carto how to visualize the point layer. BUT as I said above .. there is no sql query to make point .. so I dont know how it is working at all. I'm hesitant to change it

I just made an appeal to get some farmers to test the app.. so hopefully this could be addressed?
Error message:

An error occurred in the application and your page could not be served. Please try again in a few moments.

If you are the application owner, check your logs for details.

I believe @stlim0730 changed the process so now mapbox is creating the basemap instead of Carto.

Eventually, A user should be able to change the basemap,

For now I like
Mpabox outdoors

https://api.mapbox.com/styles/v1/mapbox/outdoors-v9.html?title=true&access_token=pk.eyJ1IjoidGF0aWFuYSIsImEiOiJjaWs1bzRiZGQwMDdjcHRrc285bTdwcWU5In0.0EWPVHyjaE9jTzNvOiIO-w#14.19/36.9835/-121.8895

or Mapbox satellite streets

https://api.mapbox.com/styles/v1/mapbox/satellite-streets-v9.html?title=true&access_token=pk.eyJ1IjoidGF0aWFuYSIsImEiOiJjaWs1bzRiZGQwMDdjcHRrc285bTdwcWU5In0.0EWPVHyjaE9jTzNvOiIO-w#12.54/36.6216/-121.5425

Since Farmview started using a different way of creating map layers to get more control on them (issue #36), Farmview has lost the on-map interaction feature provided by Carto for now.

On-map interaction that the current version of Farmview employs is infowindow on each data layer. Supports for tooltip and other on-map interactions might be added later if needed.

The initial plan was to model all the relevant Carto configuration so that an admin user can copy-and-paste the code snippets from Carto to Farmview admin. However, taking @calo1's usability concern into account, I'm trying to take plan B that modifies the application to parse (on the client side because map layers are created there anyway) the existing viz.json and extract required information and configuration. If this approach goes successfully, no more action is required to bring back all the features on Carto.

I also believe this is the right way of refactoring the code as well. This is a step forward to make this app more data-driven. Whatever the data seed is given, if the data is valid, the app will render what the data describes.

images is all garbled. needs a small fix and looks like a merge conflict

Should be Contributors instead of Who we are

To make a button to switch between languages, I am trying to use forms that go through available languages, which I have as

LANGUAGES = [ ('en', _('English')), ('es', _('Spanish')), ]
in settings.py

I use {% get_language_info_list for LANGUAGES as languages %} in the html, but the dropdown menu does not show anything.
I cannot translate to a language that django does not acknowledge the existence of.

Import id and dropbox url should not be there since they will be automatically generated from the code.

Readability is the key of sustainable project and standard-compliance is the key of sustainable web! We aim to achieve both readability and standard-compliance in this issue since they are also the two biggest issues in terms of the project's HTML code base.

The followings are the problems we address in this issue.

1. Wrong syntax of HTML; really. It exists.
   • I found several HTML elements with the same id attribute. Clearly wrong. Should have been class.
   • Also, their styles are defined per id. Clearly wrong. Should have been class.
   • There are incomplete pairs of open-close tags. Wrong and unreadable.
   • We want all the other kinds of wrong syntax gone, if any.
2. Too many inline styles
   • Inline styles are syntactically correct. However, as far as the separation of styles from data was the biggest motivation of introduction of CSS, this is not standard-compliant. Inline styles are valid only in case... almost no case. They also make the code unreadable.
   • There is always a good way to classify a group of elements and consistently style them as a group using class attribute.
     • This will be how you separate the styles from the data
3. Indentation

   • Believe it or not, the biggest factor for readability is indentation.

   • Collaborative coding process is innately patchy, so I understand there could be irregular indentation. However, we want to fix it right now.

   • We use two-space indentation per level. Note that level is in italics. Level is important.

     <grandparent>
       <parent>
         <children>
           ...
         </children>
         <children>
           ...
         </children>
       </parent>
     </grandparent>
     

   • You may leave the opening tag, closing tag, and the content in the same line if it's just a one-liner that looks fine. This helps the source code being compact. Use your discretion.

     <span class='span-example'>This is a one-liner. You don't have to add new lines.<span>

   • The same indentation rules apply to javascript.

   • Feel free to use existing formatter tools, but at your own risk. They might not be designed to properly format Django-templated HTML codes, which we're using. Also, they might not have the same rule for formatting as our local rule.

     • In my case, I rather manually go through the code to learn how it works while formatting. And then, I verify the format using such tools. Just a suggestion.
     • One of highly recommended tools in that regard is W3C Validator with Validate HTML fragment option.

Per contract Big Ideas Logo needs to be included. CartoDB needs to be changed to CARTO. Current people should get a picture. past people just get Name, title and link if appropriate. Taek's needs to change to Cornell?