mysociety / alaveteli Goto Github PK
View Code? Open in Web Editor NEWProvide a Freedom of Information request system for your jurisdiction
Home Page: https://alaveteli.org
License: Other
Provide a Freedom of Information request system for your jurisdiction
Home Page: https://alaveteli.org
License: Other
Feature allowing a user to make requests to a set of public bodies.
Needs careful consideration; perhaps needs to be developed along with a system for collecting responses (Allowing hidden links to Google Forms / Survey Monkey or similar).
Protection against mis-use would be required.
There is currently no way for a user that submits a request to unsubscribe from emails relating to that request.
"I do see that the emails such as "You have a new response",
"Was the response any good", "Clarify your request" and "Delayed
response" do not have any way of unsubscribing - is this deliberate? If
someone wants to come to the site manually to see if their requests have
updated and not get any emails, should they not be able to? Certainly we
need to provide an opt-out facility for every type of email the site
sends to comply with Hotmail's anti-spam guidelines, amongst others."
WTDK uses machine tags like ch:_
, where _
is replaced by the company number, which is eight characters long and consists of optional upper-case letters followed by digits. They are listed at http://foiwiki.com/foiwiki/index.php/WhatDoTheyKnow.com_Tags
There should be an admin-configurable way of setting the target of machine tags so they can link through to related online resources automatically.
We regularly get users writing to us saying they've entered text either as a request or a reply but can't see how to send it.
Presumably they are not realising they have to "preview" first.
Search box: Quotes from phrase search stripped out, and search does boolean AND search instead.
Steps to reproduce:
Actual results:
3. search URL returned: http://www.whatdotheyknow.com/search/safe%20and%20effective%20public%20health%20measure
=> 163 results
Expected results:
3. http://www.whatdotheyknow.com/search/%22safe%20and%20effective%20public%20health%20measure%22
=> 1 result
I think this is quite an important area to improve to reduce to overhead of running an alavateli site.
I'm making one issue rather than several different issues for the different points because I think it's best to have a single place to discuss the UI in general. Might be worth spinning off separate tickets for actual implementation if it gets that far.
At the moment, if the name of a body starts with "The", it gets filed under /body/list/t, which is unhelpful. It would be better to ignore a leading "The". Something more configurable might be better, but I can't think of any other obviously ignorable prefixes.
We should expose tags to end users (#2302), consider allowing them to add tags themselves, and use them to cross-reference possibly related requests.
Tags can also be automatically generated (for english, at least) - see https://github.com/sebbacon/alaveteli/issues/19
We can also consider cross-linking authority names where they appear in the text body, and could perhaps also geotag entries based on OpenCalais city / county data.
Some ideas here [https://github.com/sebbacon/alaveteli/wiki/Performance-issues]
Users with a public authority email address ought be able to request a re-send of any message themselves without contacting the support team.
The alert-tracks script sends out any daily email reminders that are due. The way it works (currently) is that reminders become due 24 hours after the previous one was sent. But because site usage has peaks at particular times of day, the script takes longer to run at some times of day than others. This makes it awkward to run from cron, because we don’t want to have more than one instance running at a time, but it is hard to predict how long it’s going to take.
One solution would be for the script to have an option to run in daemon mode, where it processes due alerts continually, and sleeps – either with an exponential back-off, or just until the next email is due – when there are no more to process.
Incoming emails from authorities are often multipart/alternative, and we display the text part. Often the HTML part is much more readable because of the extra formatting. Usually the writers of the emails don't even know or think about the text part and so it can be very hard to read.
Directly inlining the HTML might be problematic because it could play havoc with the layout etc of the request thread, but it would be good to use it somehow.
A user has requested an ability to print (and/or create a PDF) of an FOI request without the extraneous content in the side bar (related requests etc.).
Perhaps related to the other ways users wish to "take a copy" of a response as requested at:
https://github.com/sebbacon/alaveteli/issues#issue/11
There is a concern (from FOI officers) that the site fosters an "us vs. them" attitude which is not helpful.
It has been said that requests via whatdotheyknow tend to be less courteous that those received by other means (which means almost exclusively email).
This issue needs further research with FOI officers.
Alerts are only sent out to users when a request state changes. This means that a request that got a reply several weeks ago will only turn into an email alert when someone marks it as "successful" (or whatever).
For alerts which ask for all updates from a particular body or a particular request, we should change the logic to alert on a new incoming request, rather than a state change.
Notes from Francis:
For queries which include the status, it obviously can't do that.
Right now the alert engine uses a search query internally. Very few
alerts though are free text so there are two options:
a) For certain kinds of alerts (like on a particular public body)
special case it to use the created at rather than described at.
Continue to use described at for all free text search alerts.
b) Do as in a), but also somehow parse the free text alert.
Maybe via a tree you get back from Xapian. Use that to see if it
is referring to the status of the request, in which case use
described at rather than created at.
Might need care as to how the rest of the alerts are structured in how
they decide what to alert on and used described vs. created.
This fits in a bit with how you want to do a permission system as
well.
In the current code, I was deliberately driving lots of things through
search, which is weird to people who love relational database, but to
my mind is more general (you always need full text search in the end)
and leaves less complex code. My plan was to do permissions via the
search as well.
However, for other reasons, you might want to do more things via
database queries. But keep in mind that it might make the code more
complex when you add search features or permissions.
Sometimes, especially if a response involves a number of attachments, or is oddly formatted it would be useful if the requester (or perhaps anyone) could obtain a copy of the raw response.
One option would be a button allowing a requestor to send themselves a copy of the reply by email. The potential problem with this might be requestors replying to that message and unintentionally then taking the correspondence off-site; there would also need to be caution to avoid exposing the request addresses.
Another option would be a download of a .zip archive containing the correspondence text and any attachments.
When following links such as "I'm about to send clarification", a
form appears into which the reply can be typed. However, the
previous correspondence in that thread is not shown.
The solution is something like:
Make it so when you make followups the whole request is shown on the page.
Remove all show_response URLs, and replace with a special version of the
request URL with a new input box at the bottom and a hash link to it
A user has requested the ability to sort their Whatdotheyknow FOI requests by status, so all Successful replies are grouped together and same for Long overdue.
This kind of thing might be possible already with carefully crafted search terms; but ought be a single click via a link/button.
Anything done for an individual's own requests could presumably also be done on the authority page.
A user has let us know that when using an iphone with an external keypad to send FoI requests there are undesired breaks in the lines of text appear when displayed on WhatDoTheyKnow
An example appears to be :
http://www.whatdotheyknow.com/request/implementation_of_the_30_mile_ru
A user reported that she had received an email alerting her that a response had been made to a request for which she has an alert set up, even though the response was actually sent in September last year (i.e. nine months ago).
On investigation, what happened is:
The proposed fix is to use the created_at field rather than the last_described_at field to trigger alerts, with the exception of alerts that query on the status which should use the last_described_at field as at present, to pick up events whose status has been changed so they now match the query.
We could copy the "bounce" semantics from software such as mailman. When a user is considered to be hard bouncing, disable alerts to that user. Also alert them when they next log in that their email address is bouncing.
Requests which attract lots of comments can descend into chaos (eg OT comments, trolling, flame wars etc), so it wold be nice to have a "stop new annotations" admin function to prevent new comments from being added.
Provide a version of the RSS feeds to Include the full text of requests with links to the attachments
We are often contacted by authorities about responses miscategorised by users. In most cases the authorities are correct. We should offer authorities a direct mechanism to reclassify requests if they choose to.
I suggest it defaults to off and we enable it on a case-by-case basis, with the implicit understanding that we would withdraw it if the authority misused it.
We may need some mechanism to deal with "categorisation fights", though those can in principle happen now between users and administrators and I'm not aware of any.
Currently a multi-line censor rule doesn't work.
This means administrators often have to create tens of single-line censor rules to censor large blocks of text. This can be time consuming and annoying.
Currently the search results page states:
"People 1 to 1 of 1 for [Searchterm] "
It has been brought to our attention that this is pretty much nonsense and could be written better.
Users regularly want to send a "reply" not to the person who sent the last message in the thread, but to the main FOI email address for the public body.
eg. When the reply is an auto-responder from someone who's moved on / is away
This is currently possible, by clicking "send follow-up" at the bottom of the original request message; however this behaviour isn't made clear to users. I suggest an additional link at the bottom of the page explicitly offering the option to reply to the main FOI email address for the public body.
Sorting this out and making clear to users how this can be done would save 2-4 support requests per week, and no-doubt help more users who don't get in touch with us.
I had a request marked "withdrawn", which the public body then responded to; as a normal user my options for categorising didn't let me put it back in the status of "withdrawn".
Currently when a body changes its name the requests from both before and after the change carry the new name of the body.
Ideally requests / correspondence from before the change ought be associated with the old name; and those after the change with the new.
The site returns a 500 error, rather than a 404, for non-existent paths.
We're often asked about the ability to make requests in private, by journalists or companies.
This could be a premium, for-money feature, partly for revenue, but also to discourage private requests.
Requests could remain private permanently, or could have an embargo date on them after which we pester the user to put them public.
Some notes on the wiki at https://github.com/sebbacon/alaveteli/wiki/Embargo-requests-feature-requirements
Obviously, this feature requires much more discussion...
Payment could be handled with http://www.activemerchant.org/
Users with an email address domain matching that of the public authority domain (or users to which we've manually assigned the privilege level) ought get an improved user experience and access to more features.
(Careful to avoid giving excess privileges to those on webmail / ISP domains / those on council/gov subdomains)
Subsets of this issue are:
#37 - Allow authorities to update status
#40 - Resend request button for authorities
#231 - Allow authorities to edit their request email address
#237 - Extend information we hold about public bodies
Administrators could have an additional check-box when they make an annotation; to check if the annotation is being made in an official capacity as an administrator of the site.
This could
i/ Highlight the annotation eg. via a different background colour
ii/ Cause an alert box to appear at the top of the request thread, and on HTML conversions, and if possible on downloaded documents, suggesting the reader looks at the annotations.
Administrator annotations can be important eg. drawing attention to known inaccuracies in material released, noting where and why redaction by administrators has occurred, or explaining inconsistencies in the timeline - if admin intervention has occurred.
See https://github.com/sebbacon/alaveteli/wiki/Improved-document-conversion
Covers:
Not urgent but would make things look and feel smoother
Examples of incoming mails that aren't automatically redirected to the right request:
To: Andrew Greenhalgh <[email protected]>, FOI <[email protected]>
I'll add more in the comments as they arrive
The update-xapian-index job spits out a variety of error messages that seem to be related to the processing of attachments, specifically PDF files. There is nothing we can do about these, since we don’t control the PDF files, so there should at least be an option to suppress such messages when the thing is run from cron.
On a request page, for super users, the only admin link is to the request admin page.
It would be useful to have direct admin links to the requestor's user admin page and the relevant body's admin page as well.
This would regularly save a click / page load for many admin transactions.
On the front page of WhatDoTheyKnow.com the "Make Request" link points to the front page; the effect is that clicking it does nothing.
Given the number of users who ask us how to make a request I think this might be a problem.
I'm not sure what the solution is, but some ideas:
The tag search function only returns records with variety:authority. You should be able to search for requests made to a set of authorities that have a specified tag, allowing alerts or a feed to be set up for requests made to bodies in your local area. eg:
http://www.whatdotheyknow.com/search/tag:islington%20variety:sent
To improve request metadata (and involve user more), ask for any comments and additional information about their request immediately after they submit a new request (or possibly at the same time?)
Use case: a company that works with data on empty properties / rateable values etc makes FOI requests where they send an Excel spreadsheet to a council containing the details of the properties they're interested in and the council fills in the blanks.
The "My Requests" page has now become more of a user page.
Often we get asked where users should go to change their account details; unsubscribe from alerts, etc.
I suggest changing the text; perhaps to "My Account" or "My User Page" or something along those lines.
Email from user:
I have just made a request that included an email address in both the subject line and the body of the request: http://www.whatdotheyknow.com/request/response_times_for_enquiries_to
While I (correctly) got a warning for the email address in the body of the request, there was no warning for the email address in the subject. While this isn't a problem for me, I just thought that I would alert you to it in case you think that it is something that you need to include in your checks.
.....
Might be worth extending the check & on-screen censoring, in order to avoid spam harvesters.
After a request has been sent it takes a few minutes for it to appear in search results on the site, or on a users' "my requests" page.
This results in users wondering if it has got lost (and writing to us to ask) or in some cases making the same request again.
Ideas:
Emails are redacted from responses to prevent harvesting by spammers.
These could be revealed to all users who prove they're human via a CAPTCHA.
Currently there are regular support requests to the WhatDoTheyKnow asking for such addresses to be revealed.
Currently users get a list of their outstanding actions (uncategorised requests) when they try and make a new request or if they visit their user page.
When users have outstanding actions pending there should be a note to this effect on all pages.
(something along the lines of a flash message at the top saying "you have outstanding actions" and offering a linking to the user page where the details will be.)
Sometimes a word is mistyped in the title of a request, and if we correct it the URL changes, potentially breaking any links to the mistyped version.
I just (22 Dec 2010) hit preview on an internal review request and got a blank page; thought it might be worth reporting that as while I've not experienced it before and we do get a fair number of people reporting things along those lines.
http://www.whatdotheyknow.com/request/54822
Time 21:56 ish
We currently hide email addresses in responses, but often these are important, and the requester needs to email the team to find them out. We should display these if the person viewing the response is the original requester. (And possibly under other circumstances too: e.g. if you've earned the rights, somehow, but that's for another day. Let's make the easy fix first.)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.