fecgov / fec Goto Github PK
View Code? Open in Web Editor NEWA general discussion forum for FEC.gov. This is the best place to submit general feedback.
Home Page: https://www.fec.gov
License: Other
A general discussion forum for FEC.gov. This is the best place to submit general feedback.
Home Page: https://www.fec.gov
License: Other
programmatically access historical filings of candidates and cross reference them with a list of presidents
Right now the electronic filing RSS feed puts most of the useful information into the description
element, rendering it less than ideal for parsing into actual data. At a minimum, the FEC could put the committee ID in the title, or, as is possible, extend RSS to include an FEC-specific namespace that includes the elements contained in the description
element as individual elements of each item.
What is the best way for revision history, particularly around candidate and committee properties, to be shown? As of now, we see a few options on the table:
Are people using historical data, or is the most recent revision normally the most valuable?
Ok, so in light of the FEC user / stakeholder meetings the other week, I've got a slightly new take on the graph search I want to share. The idea here is that in addition to being able to construct a graph search or search for a specific entity, the user can also easy pull up all records for, say, candidates, and then slice it down with filters. I heard in those meetings that users want the ability to easily pull up a data set and filter down, so this responds to this.
Also these new views represent a flow that has the search results on the same search page, instead of a separate results page.
There's a couple sub-options, but both start with a basic search box and big buttons that "view all" of a particular data set.
Option A keeps the buttons in place, showing relevant filters for that data set that can be expanded.
Option B would have the page scroll down, and in so doing, the buttons and search collapse into a single persistent nav bar at the top. Everything else stays the same. (Personally, I think I like this idea most).
The FEC does a great job of breaking donation info up by candidate and by donation amount, but it is very hard to find how many unique contributors donated to each candidate, and how many people gave small vs large donations. Since only dollar amounts are shown it is not transparent to see if the >$2000 category came from a few or a lot of people. Currently the only way to find out this information is to download the raw data spreadsheet, and then manipulate to take out multiple donations by the same person to find the number of unique donors. I think this is a much better gauge of a candidates popularity and "grassroots movement" claim then the amount of money raised which is widely publicized. Please add this info so I don't have to hunt for it.
Getting FEC data into a database is difficult. https://github.com/Solomon/opensecrets_to_postgres
Since I imagine the data is originally coming from a database, can you open up your nightly database backups?
When I'm using the map to navigate to the federal candidate campaign finance viewer, the URL doesn't update in the address bar, so I can't copy and paste the link from my browser to directly link a second- or third-party to a given FEC page.
The only link I ever have in the address bar is this one:
http://fec.gov/disclosurehs/HSCandDetail.do
That's completely inscrutable! It violates almost all the rules for a good unique resource locator and greatly limits the ability for persons to navigate any part of this (name)space.
I think the public ought to be able to send someone to any and/or all of the following meaningful addresses, or a similar URL schema you design:
http://fec.gov/disclosure/ohio
http://fec.gov/disclosure/house/oh-11/
http://fec.gov/disclosure/house/oh-14/
http://fec.gov/disclosure/senate/oh/ [goes to -> disambiguation by cycle]
http://fec.gov/disclosure/senate/oh/02012 * [Citizen Fisher + Citizen Portman, Historical]
http://fec.gov/disclosure/senate/oh/02012 * [Citizen Mandel + Citizen Brown, Historical]
http://fec.gov/disclosure/senate/oh/02016 * [Citizen Portman + Challenger, Accumulating Funds]
http://fec.gov/disclosure/senate/oh/02018 * [Citizen Brown + Challenger, Accumulating Funds]
http://fec.gov/disclosure/usa/ [goes to -> disambiguation page, Map + Presidential Race]
etc.
You could even drop the "/disclosure/" term to make these URLs even more easily sharable. It doesn't need to be there, and it makes it harder for someone to sketch out the URL they want someone to visit on a napkin.
All these pages already exist — they're just seemingly impossible to directly access from the address bar, and the URL schema can be so much simpler to share from point-to-point.
RE: 0201X dates above... I like Institute of the Long Now dating, because they help remind me of 7th generation obligations, but I know of course that you'll need to drop the leading 0 on dates for publlic accessibility.
[Also, if this is covered elsewhere as a ticket, please do feel free to close this one as done. :-)]
If a citizen wishes to report a violation of a law or regulation, the Public part of the site should provide a web form to do so.
The new committee registrations are available in HTML, CSV and XML. JSON seems a relatively easy win.
I've built an interactive front end on top of FEC data, and I think it would be great if this functionality was something that the FEC itself provided!
Here's a link to the functionality of my front end: https://www.youtube.com/watch?v=_BUyIgGBNw8, and here's a link to a kickstarter campaign I'm currently running for it: http://kck.st/1DG57W4. The tool is already built (though not yet launched to the public), and if anyone at 18F or the FEC are interested in playing around with it, I will be happy to give access to a live version.
It's going to be completely open source, so I'd love to talk about how the FEC can take either some of the ideas, or just the actual project itself, and incorporate it into what you're doing.
FECFile is a Java application that's goal is to make it easy to file FEC Reports electronically. However, the application is way more convoluted than it needs to be and requires the installation of Java, so I still file by paper - which is frustrating as a programmer (my PAC does not meet the $50,000 threshold that requires electronic filing, fortunately).
The ideal experience would be to post a simple JSON object containing individual transactions (receipts, disbursements, etc.). At reporting time the application would aggregate the data, present it for final approval, and allow submission.
In terms of disclosing data, aside from the few forms that need to be filed for actions performed close to an election, there is limited transparency in terms of campaign/PAC activity except in batches as part of monthly, quarterly, or even semi-annual reporting. Allowing the submission of individual items could allow there to be near real-time reporting data available to the public.
Minor political parties and systematic discouragement
The political and social revolution begins when innocent people suffer from the Republican and Democratic policies of the Patriot Act which was made for minorities and the People Party members in an attempt to keep them in financial hardship many minor political parties are discouraged by local news media and press by not covering political and social events in local libraries or even publishing political opinion and letters. American minorities are discouraged not to take active part in political process which has been dividing America , our business shut down, house boarded up just because we believe in change. We would request freedom of the press and equal opportunities for all Americans for better future .
Sincerely,
A Mehdi
The People Party of Connecticut
507 Pine Street
511 Pine Street
Forestville CT 06010
860.583.3338
IN GOD WE TRUST WE HAVE FAITH IN HUMANITY
Sent from The People Party www.humanfealty.com
The People Party is a Minor Political party in the Constitution State of Connecticut for domestic tranquility.
Just an idea. I think people would really like to see visualizations for money spent, and contributions. Wikistream does this in real-time with wiki edits. I doubt you get the things in realtime, but I bet you have dates, and people want to see chronologically organized data. Both Megahertz and I and some others have experience with D3, which is a very helpful way to to do this.
Testing this out
It worked! (hopefully)
Go team!
Paper records provided by Senate candidates / committees are OCR'ed by an outside vendor, and the data appears in several locations: data summaries are available in the webk file (which may have changed it's name, actually, but is the file that contains one line for every periodic report filed); the line itemization of contributions are available in the FTP bulk files; and the line itemizations of expenditures are available in a third location, and typically take even longer. There's no way of knowing what data has been processed though, without checking every single file.
What would be really helpful would be a feed of which paper reports have been added to which files. The webk files, for instance, are updated daily, so the summary of the paper reports are typically available at least a few days before the contribution line itemizations are (I think more typically this delay is weeks not days, but can't say for certain). I've never measured the timeline, but I think it's often months before the expenditures from paper records are available.
My best solution would be a json feed with link to original paper record, candidate+committee id, and description of what action had been taken.
This is related to issue 11--and arguably the same--but I just wanted to highlight the special route that paper records take...
Kudos to another similar issue by https://github.com/adamclayman with
https://github.com/18F/FEC/issues/23 . (also see Tim Berners-Lee http://www.w3.org/Provider/Style/URI.html )
However, this issue is for a new feature that can change how average citizens can comment on specific elections. Having Real Simple (and Permanent) URLs for campaign orgs and candidates is crucial for finding resources (a good web site is the best ReSTful API in that it is human presentable and easier than any JSON api to extract information from). An important use of the URLs is also as a globally unique identifier and part of a great electronic citation. For this reason, citizens that want to comment on their own blogs, postings or tweets, can use the election URL to "tag" their pages with others discussing particular elections. Also, the media (like the NYTimes APIs and even in their articles) could use the URLs for searches and indexing, whether as Dublin Core/MODS/RDF/MicroDatat metadata or hyperlinks in the articles.
Also, then discovering all the candidates based by election would be easier. I would then use this as a part of rich snippets ( https://developers.google.com/structured-data/rich-snippets/ ) which would allow search and aggregators to do better jobs with your fec.gov web pages. I will make a separate issue about MicroData ( http://en.wikipedia.org/wiki/Microdata_%28HTML%29 the main format for rich snippets).
Some of the CSV files in the data catalog contain interior double-quotes, which Excel handles without error but most CSV parsing libraries do not. An example would be the Committee Summary file
Are donor and expense addresses already geocoded by the FEC and individually cross-matched to federal Senate and House Districts? If so, I have not found any clear sign of it. In the CVS exports of expenses and contributions, I can only see "City, State, ZIP". That doesn't match up precisely with Congressional Districts.
I would find it very hard to believe that Google, Microsoft, and other big players would not gladly geocode all that data for you for free, as part of their public service mission. To get permission for unlimited geocoding by the 18F for the purposes of geocoding and disambiguating FEC contribs and expenses to Senate and House Districts, I would urge you to get in touch with the Social Impact Team at Google. Anthea Watson-Strong (@antheaws) is awesome, and she's available and very responsive via Twitter. I'll try to patch her into this conversation. If she and her team can't get you access to 100,000 geocode requests per day (forward or reverse) on their own, I am sure she will know with whom you ought to be talking to get that clearance. It's the sort of thing that I doubt the Google Team would think twice about. With each contribution and expense tagged by State (which is also Senate) and House District, more important views of the data become readily accessible for end-users in a variety of organizations, and we can just pull the Congressional District and State tags rather than doing the work of geocoding and interpreting that ourselves. I would also suggest that this is well within the statutory authority of the FEC (to collect addresses on contributions and expenses and associate them with a State and Congressional District of origin) and would prove very useful for candidates who want to self-monitor their own election campaign's contribution sources, so that any funds they use remain tied to their Congressional District.
This conversation was started in a different ticket — for the start of this conversation, and an extended exchange with @dwillis and @jsfenfen, check out https://github.com/18F/FEC/issues/20.
This geocoding request is applicable most of all to the House Races rather than the Senate, since Senate contributions and expenses are already tied to the appropriate state — with State matching 1:1 with Senate electoral districts — in the publicly accessible data sets.
This could include adding basic tool tips popups to define terms or application choices, or follow-along help functionality that steps the user through certain processes.
it's the best
At the bottom of this document:
http://www.fec.gov/pubrec/fe2008/tables2008.pdf
the computation of Pres. Obama's percentage of eligible voters appears to be wrong -- I think... Using rounded numbers, the chart reports that the President received 69.5 million votes, which was 53% of the popular vote. At the bottom of the page (page 5 -- but it's the first page available?), it states that the voting age population is 225.5 million, and states that the % of the voting age population casting a vote for the President was 58.23%.
Could that be right? By my computations, I have it at about 30.82%.
I'm arguing with an idiot on Facebook & don't want to be the bigger fool... Could you please tell me whether your computations are correct, or mine?
Many thanks,
Sara Walsh
Augusta, MT.
Its on the left! Modern web design is responsive with scaling properties. If you cant afford it ill do it myself!
I took a go of some updated, higher-fidelity wireframes for the graph search. Here's a few screens demonstrating how autocomplete might work, and one screen demonstrating the results page with some filtering options.
This is still super green, but wanted to get it in front of everyone for initial feedback.
These designs are meant to address the challenge of educating the user about the possible options they can organize their search around, much the way Facebook's graph search works. And then at the results page, the goal is to present data that is easy to scan and further filter before downloading as a csv.
There's a bunch of other ux questions to tackle, but here's a start. Let me know what you think.
(And of course, these are rough wireframes, so pay no attention to the visual style).
there should be a way to search using different mechanisms -- by date, by candidate, by spender....and to easily slice and dice -- it is really hard now to do anything except scroll through by date - rather than just being able to search by race / candidate (who is spending what against / for this candidate) or by group -- it is really clunky to try and quickly look at / generate a report that shows who is spending what for or against this candidate or in this state - or how much this group has spent in a certain time period. It ought to be much more easily searchable than it is now.
A way to toggle order of contributions by all categories (i.e. comittee name, employer)
A shortcut from a contribution search directly back to advanced search
The historic reports the FEC used to compile each year should be more easily retrieved by election cycle - it is really hard to go back and find them even when I know I've seen them years back. Some better way of organizing, retrieving and searching is in order. There is alot of good historic data that has been published over the years.....if only I can find it!!
For each candidates contributor files there are entries for amounts over $2700 that are flagged with Reattribution, or other entries of a negative amount that says Refund. Looking at these it is hard to tell how many people donated, or how much because these need to be manually taken out of the file. Could the FEC publish their post-analysis file which they took these into account to get money donated by amount i.e. <$200, $200-$500, etc? Also include the number of donors, and if a donor is a repeat.
There are many non-profit organizations that regularly monitor the FEC's website but may not have seen your blogpost (or be sophisticated enough to use GitHub).
I suggest setting up meetings with :
There are also many legal practitioners who track this on a regular basis.
I would be happy to help set up some of these meetings.
If you check out the FEC Commissioners Members Page (http://www.fec.gov/members/members.shtml), you might notice that Commissioner Goodman has no contact information publicly listed at all, and the display formats given for C. Walther and C. Petersen are so different in schema from that of C. Ravel and C. Weintraub so as to make C. Walther's and C. Petersen's contact information almost unnoticeable to a quick citizen scan.
Any effort to standardize the listed contact info in a similar bubble will help prevent anyone from leaving the FEC Site under the impression that Commissioner Goodman is unreachable, or that Commissioner Walther and Commissioner Petersen (who post contact information, just in a different format) are too.
Are the forms Schedule A, B & C from the IRS (for 527 orgs) related at all to these documents being mentioned in openFEC?
Each district across the country has specific additional requirements imposed beyond the Article I, section 2 requirements. E.g., petition signatures. These should be listed under the candidate's information and in a data section of the site.
Also, the FEC should obtain voter information from each whole state and provide this to each person who registers as a candidate for federal office.
The FEC should obtain voter contact info, including email address, for each state, for federal candidates.
The FEC should create an official petitioning system that all federal candidates can use to be put on the ballot. This system should have an iPhone app associated.
Every voter should be allowed to set up an account and interact with candidates. Voter status should be verified against the data from the states. This encouragement of 1st Amendment activity is very appropriate.
Again, thank you for the opportunity to make suggestions.
Andrew Straw
www.andrewstraw.com
APIs for the data and services that make up FEC.gov is the single best step that the agency can take! That way, more developers can provide a better user experience for different users.
DEMOCRACY : The legislative and executive power in any democracy of the world has right to direct how force of democracy should be employed for preserving the community and members of the branches.
The matter of The Patriot Act is that it exempts themselves from obedience to the laws they make in both its making and executing to their own private advantage. Thus to me this is the end of Democracy. Therefore a question arises that in a well ordered democracy where the good of the whole society is so considered as it ought, the legislative power is put into the hands of diverse minorities who suffer from the law.
In any democratic society laws are ought to be designed for the good of all citizens. Such as raising the taxes of property or eminent domain laws with the consent of the Citizens.
The fundamental law of nature that represents Freedom or free will is based on Democratic values. God or nature never allows a man to abandon himself. Human Beings are not allowed to take their own life neither this power given to State legislative or executive powers.
Then why is The Patriot Act in a democratic society. Thus, innocent people have no other remedy in this law as in many other laws of this new American Democracy.
The fundamental law of nature has thought man that where they have no Judge on earth or justice in democratic society then appeal to heaven for the rulers in such matter for injustice.
By reading and understanding the new rules of American Republicans and Democrats legislative and executive power this is absolute monarchies here as well as other governments who copycat our laws.
Sometime I wondered, God having made Man with such an intelligent mind that in his own judgment it was not good for him to suffer and live in society, with different languages, cultures , family , justice and freedom of free well. We would request FEC to help minor political Idealogies for Change and different America that stand for Freedom and justice .
Sincerely,
A Mehdi
The People Party of Connecticut
www.Americansparty.com
My name is attorney Andrew Straw and I have worked on court record reform in both Virginia and Indiana. When I worked on the Indiana Supreme Court's new website, I suggested that the Court use a user-oriented format. This made the entire Indiana Supreme Court website easier to navigate, with less wading through irrelevant material and links.
Note how the Court has the categories I came up with at the top:
http://www.in.gov/judiciary/supreme/?vm=r
If I were advising the Court today, I would suggest that they make the font bigger and compliant with the ADA in every way.
The first thing I would recommend to the FEC, therefore, is to reorganize the information on your website so that it allows different types of common users to find the information they need easily. These categories need to be there:
Candidates (President, Senate, House)
Attorneys
Treasurers
Public
Students
Data/Search
Press
Each of these should have its own FAQ and FEC staff contacts, with both fax and email.
Every type of user should be able to have an account. Public and Press accounts should have the ability to track different candidates and their filings, with notifications. There are so many ways to provide outstanding services by user type.
If there needs to be some cross-referencing within each of these categories, that is perfectly fine.
When someone comes to your site, they should see very clearly a category that describes why they are visiting. They should not have to sift through lots of links and URLs to find what they are looking for.
Also, there needs to be an online chat assigned to each type of visitor. Many private businesses and government offices have this, and having a chat service makes the entire site 100% friendlier to use. People should not be required to call to get information, but calling should be an option. The chat service should go from 9am to 10pm EST M-F, and from 1pm-8pm on Saturdays and Sundays.
You should outsource part of the chat service. You should offer the chat in several languages, including English, Spanish, French, Russian, Chinese, and Arabic. Every chat operator should be native in the language in which they are providing the service, and fluent in English also.
Thank you for the opportunity to comment. If you would like to contact me, my email is [email protected]
Sincerely,
Andrew Straw
www.andrewstraw.com
Would it be possible to allow the download, or put on the ftp site, candidate summary data from pre-2008, in the format you have for 2008 and onwards?
If not, could I get some guidance around how to figure out unitemized donations from before 2008?
I tried using the space-delimited files for can_sum on the ftp site, but there doesn't seem to be a way to pull the small money donation amount from that file format.
I've also tried subtracting all the individual donations from the total individual donations from the candidate summary, but the numbers don't match up for the following reasons: 1) the bulk data individual contributions don't include contributions from LLC's and LLP's, which can significantly affect the total donations, and 2) small donations from donors who already gave previous reported donations are not included. So, if I gave $2,000, and then the next day I gave $50, that $50 is not included in the bulk download.
New issue for new wireframes.
First, a sketch of the various flows that different searches would take through different views.
And here's screens for how the autocomplete can work. The challenge here is to explain the grammar and logic of the searches. I'm proposing we develop a set of icons and colors (doesn't have to be these ones) to communicate a matrix that includes the three types of records: Transactions, Candidates, Committees; and the three dimensions that each has: Type -> Properties -> Values.
When looking through committee reports that contain joint allocated federal/nonfederal activity in HTML format, any attempts to open Schedule H4 result in the following:
ORACLE error-- ORA-00904: "DESCRIPTION": invalid identifier :
Example: http://docquery.fec.gov/cgi-bin/forms/C00010439/1010437/h4
Currently the only way to read Schedule H4s is in PDF format, which makes research quite a bit slower.
Is this something that can be fixed?
Minorities like us, which is less than 1 % in the United States, continue to face serious threats, discrimination and racism, and are frequently excluded from taking part fully in the economic, political and social life. Today, we face new challenges, including legislation, policies and practices that may unjustly impede or even violate our rights by taking our Proprties, closing our Bussiness and keeping pressure and problems so we would give up our dream to change America for Freedom and Human Rights.
The People Party of Connecticut
www.humanfealty.com
We have some endpoints that we would like to display in a visual, interactive way but are still developing ideas. Maybe it's best to do a choropleth map, or something more creative.
I outlined some of the good visualizing datasets in this post:
https://18f.gsa.gov/2015/07/15/openfec-api-update/
There are a lot of good political results maps and visuals, but we are focusing on the campaign finance data.
Thanks!
On WWW.fec.gov/pages/brochures/citizens.shtml it says $2,700 limit for individual contrib but lower down it says under Contributions from Family Members .....For example a couple may contribute a $5,200 check........ I think this should be $5,400, which is 2X$2,700. Yours truly, pgra ([email protected])
For polling use, it would be helpful to have a single URL that just returns the maximum / most recent .fec ID number for regular and senate filings respectively (as in http://docquery.fec.gov/dcdev/posted/#{filing_id}.fec
and http://docquery.fec.gov/senate/posted/#{filing_id}.fec
).
The FEC sends emails re: updates re: Advisory Opinions. When you click through to see what it is, the link takes you to the search page. So unless you actually wrote down the number of the AO or AOR which is the subject of the email, you have to go back, look it up, and then go search for it. The search engine to FIND / research advisory opinions is so much more difficult than before it was 'improved' several years ago. It is almost impossible to find advisory opinions even when I know there have been opinions on that subject. So the entire search process for advisory opinion research needs to be completely overhauled -- the links that the FEC sends out about updated AO information should take the user directly TO the subject of the email without having to go through a search -- just link to it for pete's sake. Then, a complete overhaul of the search engine for finding AO's needs to be undertaken -- go back and look at what was discarded when it was 'updated' a few years back. THis is so much worse than it used to be. Finally, it is not possible to read AO's search for on an iPad. It is impossible to navigate on newer devices supported by other systems such as research on an iPad. This is a very clunky and not user friendly system. It was better before it was 'fixed' so I've yearned for the old system. It was so much easier.
When you look at a Candidate Summary from fec.gov, and compare the list of individual contributions from clicking on the link to the individual contributions from the data dump, they are not the same.
In particular, it seems like a bunch of law firms and LLC's are included in the candidate summary view, but are not included in the bulk data. I would rather include them in the individual contribution bulk data, because then it becomes possible to correctly calculate unitemized contributions to a candidate, but either way, I think they should match.
Second question, are those LLC and LLP contributions located anywhere in the bulk data?
Feedback box testing by Wei. Please ignore
learning about campaign finance!
Most of the FTP data files are updated every week early Monday morning, but it would be useful to have some kind of feed or page that provides an "as-of" date/time for data updates, including the data catalog. Another option would be to provide a "last updated" timestamp on pages like the maps.
There are quite a few websites that attempt to make FEC data more transparent and approachable. But (almost?) all of them focus exclusively on the contribution side. And while transparency about contributions is good, there are very strong reasons to make expenditures transparent as well. Among those reasons:
Sir or ma'am:
In the face of rampant and blatant attacks on voting rights by Republicans, I highly suggest a new method of online voting through the Federal Election Commission, or other suitable government agency, be created and implemented to help ensure that EVERYONE has a chance to vote. Thank you!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.