jyby / repositorium Goto Github PK

View Code? Open in Web Editor NEW

4.0 9.0 5.0 5.23 MB

Document bootstrapping database with collective quality control

PHP 97.92% Shell 0.01% Perl 0.65% JavaScript 1.41%

repositorium's Introduction

Repositorium

Web Engine for Collective Repositories. Based principally on CakePHP and MySQL

Initiated and mainly directed by Jérémy Barbay. Participating students listed in Section (Academic) History

Installation

Requirements

MySQL 5.1
PHP 5.3 (any php5 should work fine too)
Apache 2.2

Getting the source

Open a console and get the source

git clone git://github.com/mquezadav/repositorium.git

Configuration

Then you’ll have a copy of the source at local.

Create the following folders and give these permissions (some of them already exist):

chmod o+w app/tmp
chmod o+w app/tmp/cache
chmod -R o+w app/tmp/cache/persistent
chmod -R o+w app/tmp/cache/models
chmod -R o+w app/tmp/logs

Copy database.php.default to database.php, and core.php.default to core.php, and bootstrap.php.default to bootstrap.php in app/config.

Edit database.php:

var $default = array(
    'driver' => 'mysql',
    'persistent' => false,
    'host' => '<HOST>',
    'login' => '<LOGIN>',
    'password' => '<PASSWORD>',
    'database' => '<DBNAME>',
    'prefix' => '',
);

Change <HOST>, <LOGIN>, <PASSWORD> and <DBNAME> (database name) to the correspoding values in your system

Initial DB Dump

To load the dump file (repositorium.sql) with initial data (users: admin and anonymous). It contains no documents and no criteria.
mysql -u <LOGIN> -p <PASSWORD> <DBNAME> < repositorium.sql
Now you have 2 users: anonymous and admin. The admin user has the following atributes
- Login: [email protected]
- Password: password
Then add a criteria, and then some documents or users.

Security

At core.php file (app/config/), at lines 204 and 209, randomly modify some alphanumeric characters of the corresponding Salt and Cypher Seed. Example:

Configure::write('Security.salt', '<SOME LONG RANDOM STRING>');

This changes’ll avoid keeping the same session at different app clones, and for security sake.

(Academic) History:

2011A: Initial project
- Project for course “CC5401 - Ingenieria de software”

Departamento de Ciencias de la Computacion (DCC), University of Chile

Staff:

Jérémy Barbay -jyby@github (course client)
Hernan Fierro (project manager)
Pablo Estefo - pestefo@github (analyst)
Felipe Banados - fbanados@github (designer)
Nicolas Perez - thomaslight@github (developer)
Mauricio Quezada - mquezadav@github (developer)
David Contreras - dcontrer@github (tester)
- Object:

creation of the first version of the prototype:
- only text fragments
- simple parameterized quality control
- one single repository per server
1. 2011A: Multi community
  - Internship
  - Staff:

Jérémy Barbay - jyby@github (advisor)
Mauricio Quezada - mquezadav@github (developer)
- Object:

Clean the code and correct some bugs
Extend the engine to support multiple repositories, each with its own parameters and url.
install an instance of the engine at http://repositorium.cl
1. 2011B: Repositorium for Students
  - Internship
  - Staff:

Jérémy Barbay - jyby@github (advisor)
Carlos Gajardo - cgajardo@github (developer)
- Object:

Add support for file attachments
fill a repository with education files (e.g. pdf of past exams) for students, inspired by the project Sensei (now collapsing for lack of participation)
propose to some students to use the new repository
survey student usage of the new repository
1. 2011B: Repositorium for Professors
  - Internship
  - Staff:

Jérémy Barbay - jyby@github (advisor)
Hernan Fierro (developer)
- Object:

correction of various bugs
implementation of the bug and feature management
Add support of visualisation features (e.g. LaTeX, HTML, etc…)
fill some repositories with pedagogical material (e.g. solved problems to compose assignments and exams) for professors, inspired by existing ad-hoc repositories (without quality control nor incentive to contribute)
propose to some professors to use the new repositories
survey professor usage of the new repositories.
1. 2011B
  - Project for course “CC5401 - Ingenieria de software”

Departamento de Ciencias de la Computacion (DCC), University of Chile

Staff:

Jérémy Barbay -jyby@github (course client)
César Campos -ccampos@github (project manager)
Rodrido Peralta -(analyst)
Leonardo Rojas -LeoRojas@github (developer)
Matias Bensan -(designer)
Rafael Meruane -(tester)
- Object:

Detection and management of duplicated material
- Main aim to counteract the simplest attack consisting to resubmit exact copies or minor variants of documents already in the repository.
- DOES NOT aim to implement advanced “plagiarism” detection.
Added tag-cloud.
Search for title and contents.

repositorium's People

Contributors

Stargazers

Watchers

Forkers

hfierro cgajardo rmeruane lmun

repositorium's Issues

User Credit Transfer

Let a user give some of his credit to another one, through email (which can be used as an invite as well).

For instance, this could be used by a professor to give some credit to his TA,
or for a TA to give back to his professor (some of) the credit he won by uploading the professor's problems.

ADD a link to github.com/jyby/repositorium/issues/new

Add a link to github.com/jyby/repositorium/issues/new at the bottom of all pages of the interface of repositorium, so that people can add bugs there if they find any.

Better choice of Criteria in Challenges

Currently, in challenges a random criteria is chosen, then a challenge
is filled with validated and non-validated documents until it reaches
the size specified.
The feature consists in choosing a criteria such that there are enough
validated documents (currently, if not, it only fills with non-validated
documents).
In search, if there aren't any criteria such that it has enough validated
documents, display a warning saying that the quality of the documents
for download will be low.

=Threshold Security=

Add a parameter $t$ to each repository,
- let the repository administrator choose and edit the value of
  $t$, by default set to $1$ (i.e. no threshold)
- if less than $t$ documents match a query,
  - the request is refused, meaning that
    - no challenge is sent
    - no document is delivered
Reason:
- the subset given in answer to a query is a random subset, so
  that to make exponentially expensive to download all
  documents of a repository (and duplicate it elsewhere).
- by using queries precise enough (this can be automatized), one
  attacker can make disappear the randomness (unless many
  documents have exactly the same tags, but then the search
  needs to be improved).

IE8 comma bug

Tags cannot be added when using IE8, in addition typing comma has no effect.

Criteria too long badly displayed in rolling menu

A criteria which is too long displays in rolling menu without troncation...

In documents browsing (All Documents, Validated Documents, Pending
Validation), one should be able to filter the documents so that to
see all documents
- submitted by a particular user
- evaluate by a particular user (?if possible at all?)

Feature for Administrators to Invite people to Repository

In the administrator menu, when adding a collaborator who is not yet registered, the following error message is generated: /Error: Collaborator was not added. Invalid email or that user is not registered/

The two cases (incorrect email or unregistered user) should be separated and given separate error messages
In the case where the user was not registered, the option should be given to send an email to this address in order to invite the person.

ALLOW users to enter a "justification" of their answer

ALLOW users to enter a "justification" of their answer.
ALLOW Experts to reward it if useful to their final decision.

The problem will be how to visualize the justifications if there are too many: those are more difficult to combine than votes, yet not impossible to do. I would imagine a cloud representation of the words used as justification, with words used more often written larger.

JPEG inclusion/Visualisation in input text

An example is in https://ilt.seas.harvard.edu : The web server
displays mere text with inclusion of images.

Duplicate detection

Define a distance between documents, automatically computed
- keywords
- content
When someone submits a problem or question,
- perform an automatic check to see if there is a similar
  problem already in the database.
- add a challenge to ask other users to validate that this
  document is indeed distinct from similar previous one.

Replace the logo by one which we know is open source

Maybe a hand drawing of the book with some stars,
or something with owls as I previously planned.

Split Repository

Let the administrator of a repository
- select some documents and
- create an other repository seeded with those,
- removing those from the current repository
The idea is that as a repository grows, some might want to split
it in smaller ones.
One must think how the points of the users of this repository are divided between the two new repositories: the simplest would be to let the admin simply duplicate the points (as long as points cannot travel between repositories)

Make Repositorium's pages indexable by Google (and other web engines)

At least the description of the content of each repository (and the future list of public material for each) should be indexed by web search engines, to emulate the same kind of behavior than stacks-exchange.

Unexplicit Error Message

When entering a new document (the 5ths in a sequence), I got the error message
"There was an error trying to save the document. Please try again later"

I would like to get a more explicit error message, if only to help debugging...

PAY money instead of time

Allow users to

buy points for credits
buy a package of documents for some fixed amount

Deletion of Criteria

When deleting a criteria, and creating a new one
documents validate for the old criteria are validated for the new one...

Sign-in vs Sign-Up

The term "Sign-in" is confusing: some users (me) will confuse it with "Login"
On other websites, is used "Sign-up" instead.

SETUP default values for the security parameters at the creation of a criteria

Currently the fields are empty, and one fills them with ones when in a hurry to create a criteria.
Instead, one should have the fields already filled with default values which make sense.

For instance:

(a,b) for a successful challenge = (0,-1)
(a,b) for a failed challenge = (2,0)

Recycle bin

Users, documents, criteria, etc. have a field =active= in the database,
meaning that that entity is 'deleted'
In cakephp, models can be filtered by a query saying that only fetch results
that have =active= with a certain value
Implement the views and controllers to manage this feature in the UI
- A trash of entries partially deleted, with an option to remove them definitively or
  restore from trash

"Invalidate" button

A button with text "Invalidated" is shown in the "Manage Repository" interface tab "All Documents". Two buttons, one for Validate and one for Invalidate should be shown instead.

Allow an expert to upload a batch of files

The first thing that the owner of a collective repository does is to seed it with a bunch of documents, or to ask some experts to seed it with some documents, for which upload nobody pays points (there is no document to validate yet in the repository anyway).

Doing it one document at a time is a pain, it would be nice to have a tool to upload a bunch of files in one single action, for instance through a tar file or a nicer graphical interface. For text documents it could be done simply by uploading a single text file containing the concatenation of all text documents with a parameterizable separator.

Show Tags of each document during challenges

Show Tags of each document during challenges, so that the quality criteria can refer to the tags and their relation to the document:

the tags are adequate
the tags correspond to the document

Open ID login

Make it possible to use another account (e.g. UPasseporte at University of Chile, Gmail in general, other...) to identify oneself in repositorium.

LateX visualisation in input text

It's already possible to add LaTeX text as a document,
but it would be nice to visualize it directly in the browser,
either automatically or through a "preview" button,
such as what is done on http://cstheory.stackexchange.com/

PDF/JPEG upload of documents Costing Points

  - System to upload a digital document similar to the cost of
    uploading a new text document in that it asks to spend points,
    but
  - the cost in points should be proportional to the *size* in MB of
    the document, so that to encourage the use of links to external
    storage.

Document Requests

 - User can specify tags and request documents satisfying those tags, giving points for their request.
 - The first users to upload a document matching those tags, receive the usual points and in addition the bonus put by the original requesters (once the collective quality control system validated the pairing of the tags with the document, and the quality criteria. )

=Correction Submission= and =Versioning=

allow to submit corrections, which will be evaluated
when detecting that some documents are very similar, propose to
submit a new version rather than a new documents.
give incentive to upload a new version (benefit partially from
the reputation of the original document?) as opposed to submit a
new document.

Let user "Flag" documents

complementary to all other evaluations
permit to detect errors not in the quality criteria
costs small amount of points, and rewards with some points when validated
require new menu(s) for expert/admin
- to list and validate flags
- to parameterized the cost and reward of a flagging event
  (maybe ask for the award amount each time a flag is validated?)

Crash when changing "Criteria" in Edit Document page

When editing a document of the repository tcspbs, selecting another criteria than the one already selected results in a page with the following error message:

Not Found

Error: The requested address '/manage/edit/49/13' was not found on this server.

Security Profiles for Criterias and Repositories

In order to help an admin to choose the parameters of security
of the repository or of the criteria (e.g. constants in linear functions)
propose various profiles (e.g. paranoid, trusting, etc...)
corresponding to predefined sets of values.

Correct labels while entering a new document

When entering a new document, adding the labels one after the other, it seems impossible to edit a label previously entered, which forces the user to delete the label with an error and then add its correction.

(I almost put it as a bug)

Why URLs should be limited to at least 4 symbols?

Why does the engine refuse url names such as "tcs.repositorium.cl"?
I see no reason to refuse those, especially to those early users?

NON Boolean challenges

where the client enter number or text rather than just a yes or a no answer to a criteria question
it is harder to check, but still possible: giving the positions of typos in the text
Problem: it is harder to represent to the expert

SETUP Validation of Tag length

A server error is thrown and shown when sending documents with long tags.

Solutions:

Tag lenght should be allowed larger
Tag length is not beign validated in client side.

Make parameterizable the "Yes" and "No" answers

let the expert enter the text replacing "Yes" and "No"
This will allow for instance to put questions such as
- "The graph presented is Acyclic/cyclic"
(not too important, but it will open the way to multiple choice
challenge questions)

=Renting= Pay points to block a document

pay an amount of points (or money?) to block a document for a
given period of time, for instance to use it in an assignment or an exam...

Add a third category of document, "PUBLIC"

In addition to the categories "to be evaluated", "validated" (for each quality criteria),
add a category for the whole repository such that administrators can mark documents which

are a good publicity for the content of the repository
have already "paid" for themselves acting as a prized

In addition, when an administrator stops managing a repositoriy, he can make all the documents "public" to make them freely indexable and available.

List of Submitted Documents in User Interface

A user who submitted documents will wish to see feedback and stats on his submissions,
and a list of past accepted and refused submissions.
This will help also new users to learn what is an acceptable document and what is not.

When entering a " " tag, it generates an error (with a non explicit message)

Solution(s):

remove automatically all tags composed of white space
improve the interface to avoid capturing such white tags anywqy
at the very least make the comment more explicit.

Option for Owner of Repository to export content

The owner of the repository should have the option to export the content of the repository.
A complete interface would allow to select a set of documents (e.g. documents with all criteria validated) and to permit the download of a compressed tar ball containing all those documents.

(This is distinct from the database dump that the administrator of the website can currently do).

Given that this is one way for owner of repositories to make money, maybe there should also be an option for the owner of the website to charge some money to allow the export of the data.

Feature for Users to invite people to Repository

When a user has some points in a given repository, he should be given the options to

send an invite by email to a friend to invite him to participate to this repository
give some of his points to his invited friends in order to ease his introduction (investment)
(OPTIONAL) Receive a percentage of the points won by his friend in this repository in the future

=FREE= Sample documents on Home Page of repository

  - ADD boolean property "SAMPLE" for each document
  - Allow experts to edit this property for each document (manually)
    - Design an interface to help experts to see which documents are
      of high quality, and "have paid for themselves" so that they
      can select them and make them free
  - Display on the home page one random "SAMPLE" at each load, as a
    way to advertize the type and (high) quality of documents in
    this database.

SEPARATE =Earned points= from =Bought or Given points=

so that submissions requires exclusively =earned points= (one cannot
"contaminate" a repository by paying sufficient money, in order
to then answer challenges on its own documents)
while downloads can be paid by both =earned= and =bought or
given= points.

OpenID support

Add support for OpenID login, so that people do not have to remember yet one more password...

NO CRITERIA = NO DOCUMENT

 - When there is no criteria, yet some documents in the repository
 - no documents show in "All documents"
 - Probable cause: it requires a criteria to sort the documents?

MOVE to variables all the text appearing on the website

so that it can be translated easily in other languages,
either at the level of the interface of the owner (which would be ideal) or at the level of the server (which I guess would be easier).

Switch density of documents

When watching a list of documents as an expert, clicking on a switch on/off for the density view which alternate between
the view

     - 1. answer one, 2. answer two

and the view

     - 1. answer one
     - 2. answer two

Add a way to contact the administrators of a repository

Context: When browsing a repository, I may want to contact its creator or one of its administrators, in order to ask to be added as a collaborator for instance.

Solution:

At the creation of a repository, require a contact email address (by default, the email address of the user creating the repository, but giving as an option to specify a distinct email for each repository)
When displaying the information of the repository, give a mailto link to write to this contact email.