houdiniproject / houdini

Free and open source fundraising infrastructure for nonprofits and NGOs

Houdini

Note: This is the latest version (pre-2.0) of Houdini and is currently in HEAVY development. You may want to use v1 instead.

The Houdini Project is free and open source fundraising infrastructure. It includes...

• Crowdfunding campaigns
• Donate widget page and generator
• Fundraising events
• Nonprofit Profiles
• Nonprofit payment history and payouts dashboard
• Nonprofit recurring donation management dashboard
• Nonprofit metrics overview / business intelligence dashboard
• Nonprofit supporter relationship management dashboard (CRM)
• Nonprofit org user account management
• Simple donation management for donors

The frontend is written in a few custom frameworks, the largest of which is called Flimflam. We endeavor to migrate to React as quickly as possible to increase development comfort and speed.

All new backend code and React components should be well tested.

Supported operating systems

• Ubuntu 18.04, 20.04, 22.04 or equivalent

Prerequisites

• Node.js 16 (we require 16 because we want the full internationalization built-in)
• Yarn
• PostgreSQL 12 (10 probably works)
• Ruby 2.7
• Ubuntu 18.04, 20.04, 22.04 or equivalent

Note: All tools will be installed in the Dev Setup.

Get involved

Houdini's success depends on you!

Join our Discussions chat

[https://github.com/houdiniproject/houdini/discussions]

Help with translations

Check our translation guide to translate Houdini to other languages.

Help with usability tests

Check on contribution_guide_usability_testing.md and create an issue with your test design or run test sessions for opened usability testing issues.

Dev Setup

Installation prep

Houdini requires a few pieces of software be installed as mentioned in Prerequisites above, as well as some optional pieces which make development much easier. The optional tools include:

• RBENV - rbenv is a version manager tool for the Ruby programming language on Unix-like systems. It is useful for switching between multiple Ruby versions on the same machine and for ensuring that each project you are working on always runs on the correct Ruby version.
• Automatic Version Switching for Node (AVN) - similar to RVM, AVN makes it simple to switch between versions of Node. When properly configured, it automatically switches version at the console when you change to a directory for a project prepared for AVN, like Houdini.

One-time setup

Postgres installation

You'll want to run the next commands as root or via sudo (for Ubuntu 18.04 users or anyone running ProgresSQL 10, change "postgresql-12" below to "postgresql-10"). You could do this by typing sudo /bin/sh running the commands from there.

Curl install

apt update
apt install curl -yy

Node and Yarn install

curl -sL https://deb.nodesource.com/setup_16.x | bash -
curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add -
echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list
apt update

Postgres install

apt install git postgresql-12 libpq-dev libjemalloc-dev libvips42 yarn -yy

You'll run the next commands as your normal user.

Note: in the case of a production instance, this might be your web server's user. Note: We use RBENV inside the project folder to have more control over the exact version of Ruby. Tip: To get out of the root shell, run exit

Run the following command as the postgres user and then enter your houdini_user password at the prompt.

Note: For development, Houdini expects the password to be 'password'. This would be terrible for production but for development, it's likely not a huge issue.

Create user account for the database connection

sudo -u postgres createuser houdini_user -s -d -P

Now that we have all of our prerequisites prepared, we need to get the Houdini code.

Cloning project

git clone https://github.com/HoudiniProject/houdini

This will download the latest Houdini code.

Let's run the Houdini project setup and we'll be ready to go!

Get the latest rbenv

git clone https://github.com/rbenv/rbenv.git ~/.rbenv

Add rbenv to bashrc

echo 'eval "$(~/.rbenv/bin/rbenv init - bash)"' >> ~/.bashrc

Note: close and reopen your terminal.

Download the rbenv install feature

git clone https://github.com/rbenv/ruby-build.git "$(rbenv root)"/plugins/ruby-build

Ruby install

rbenv install 2.7.8

Setup project

cd houdini
bin/setup

Note: The .env file holds your environment variables for development; on production you might have these set somewhere else other than this file. Tip: On Heroku, the environment variables are set in your Dashboard.

Also, you should set the STRIPE_API_KEY and STRIPE_API_PUBLIC environment variables which you'd get from the Stripe dashboard. On your development environment, make sure to use test keys. If you don't, you're going to be charged real money!

Stripe keys setup

Get Stripe keys

Go to Stripe, create an account or just log in with you already have one. Access the stripe dashboard and copy both publishable and secret keys.

make sure to use test keys. If you don't, you're going to be charged real money!

Configure the .env file

Then after retrieving both keys copy them into your .env file on these lines:

export STRIPE_API_KEY='REPLACE' # use your test private key from your stripe account
export STRIPE_API_PUBLIC='REPLACE' # use your test public key from your stripe account

Testing

To verify everying is set up correctly, you can try running through the Ruby test cases:

bin/rails spec

(You can also run bin/rspec, which provides the full feature set of rspec such as --next-failure and --only-failures, but does not set up a test database when it doesn't exist.)

You should expect to see the output of the test execution, including messages about pending test cases, and eventually get the output to the effect of below:

Finished in 6 minutes 25 seconds (files took 10.35 seconds to load)
2433 examples, 0 failures, 42 pending

Coverage report generated for RSpec to .../houdini/coverage. 10552 / 12716 LOC
(82.98%) covered.

The important thing to look for is that the number of failures is zero.

We also recommend you run through the javascript test cases by running:

yarn test:js

Lastly, you can use Storybook to experiment with the various new React components.

yarn storybook

If you create a new React component, make sure you add a storybook and jest tests for that component!

Creating your first nonprofits and user

To create a nonprofit, use the command line to run the following command and fill in the questions with the required information:

bin/rails houdini:nonprofit:create

There are available arguments that add configurations on the nonprofit's creation:

# Make the nonprofit admin a super user (they can access any nonprofit's dashboards)
  -s, [--super-admin], [--no-super-admin]
  # Autoconfirm the admin instead of waiting for them to click the email link
  # Default: true
      [--confirm-admin], [--no-confirm-admin]  
                                               

Additionally, it is possible to provide arguments to fill in the fields for the nonprofit creation without coming across the questions:

      # Provide the nonprofit's name
      [--nonprofit-name=NONPROFIT_NAME]
      # Provide the nonprofit' state code      
      [--state-code=STATE_CODE]
      # Provide the nonprofit's city                
      [--city=CITY]
      # [OPTIONAL] Provide the nonprofit public website 
      [--nonprofit-website=NONPROFIT_WEBSITE]
      # [OPTIONAL] Provide the nonprofit public email
      [--nonprofit-email=NONPROFIT_EMAIL]
      # [OPTIONAL] Provide the nonprofit's 's phone
      [--nonprofit-phone=NONPROFIT_PHONE]
      # Provide the nonprofit's admin's name
      [--user-name=USER_NAME]
      # Provide the nonprofit's admin's email address(It'll be used for logging in)
      [--user-email=USER_EMAIL]  
      # Provide the nonprofit's admin's password              
      [--user-password=USER_PASSWORD]          

You can use this in the future for creating additional nonprofits.

Startup

bin/rails server You can connect to your server at [http://localhost:5000]

Super admin

There is a way to set your user as a super_admin. This role lets you access any of the nonprofits on your Houdini instance. Additionally, it gives you access to the super admin control panel to search all supporters and nonprofits, which is located at /admin url.

To create the super user, go to the rails console by calling:

bin/rails console

In the console, run the following:

admin=User.find(1) #or the id of the user you want to add the role
role=Role.create(user:admin,name: "super_admin")

Code Analysis

We use Rubocop to perform static code analysis:

rubocop

Additional documentation

We have some additional documentation describing some implementations, definitions and other guides on the docs folder.

Known issues

For a list of how to solve known issues

Run in production

You will likely want to make a few changes in your configuration of Houdini before running in production as you would for any Rails project. For details, see production deployment.

houdini's Issues

Stripe Connect Validation process doesn't automatically set to escalated if necessary

Occasionally, a Stripe Connect account will require an elevated validation process. Stripe requires this when there's a mistake in the DOB or the last four digits of the SSN for the person setting up the nonprofit account. Houdini has support for turning on an "escalated" verification mode which allows the nonprofit user to enter their full SSN for verification. This requires setting Nonprofit.verification_status to escalated. While we do update the verification status through a cron job (and it could be done through a web hook), we don't currently have an automated way to set the verification status to escalated. For CommitChange, I manually set the escalation status via the rails console when we find out it's necessary. This is not ideal :)

Adding support for updating to escalation status requires the following features:

• sensing the need for escalated verification during the verification status cron job and setting Nonprofit.verification_status to escalated
• Email the Nonprofit user that they need to come back to the verification page and do the escalated verification process

Add Donation IDs to Refund Displays

Expectations: When a user makes a refund, they expect appropriate ID numbers to display on the refund record.

User Story: When San Mateo PAL (a CommitChange nonprofit) had to refund some duplicate charges, it was confusing as to which refund they had already made.

Resolution 1: Add Donation IDs to the refund display.

Address standardization (use case: international events)

User story: "An organisation needs to be able to setup and manage events that are based outside of the United States, but Houdini's existing address fields do not allow for locations without US States. To solve their problem we need a solution that standardizes address collection without losing well-structured data."

Overview: Addresses throughout Houdini should be assessed and restructured to accomodate as many international use cases as possible. The most pressing features for internationalisation relate to events.

Tag export for supporter exports

Expectation:

When a user creates an export from Supporters, a field with tag information will be included in the export.

Reality: When a user creates an export from Supporters, that information is not included unless a custom report is created by a dev.

Resolution 1: Create the option to add tags to a Supporter export (like a checkbox that comes up near the Export button)
Resolution 2: Add tags to all Supporter exports.

Make onboard page optional

In some cases, a organization may never want to make onboarding possible. This is particularly the case for instances where there's a single nonprofit with no others. To this end, we need a setting that allows instances to not have to onboarding available.

React-ify the App

Houdini Project's front end will be rewritten in React. The rewrite in React will have the following characteristics:

• Written in Typescript (.ts and .tsx files)
• Heavily tested using a test framework (TBD which one)

Long term project.

Have landing page redirect to onboard or login

The current homepage has basically no purpose. I propose we have the landing page simply redirect based on the following:

• If there is a registered nonprofit already, redirect to the dashboard page.
• If there isn't, redirect to the onboard page.

Better Sorting for Campaigns

Expectations:

Users expect to be able to sort the display for Campaign dashboards by different data points to have a better view of data at a glance.

Reality:

Currently, you cannot sort campaign dashboard displays by the data present. For example, some of our users want to be able to sort by campaign gift levels, so they can see the info on their screen at a glance without having to make an export.

Support for Multiple Addresses

Via Eric: Currently, you can only have a single address for a supporter. This causes problems for shipping addresses replacing the default address, as well as other cases where the address is different. The multi address support needs the following characteristics:

• Support for multiple addresses on one supporter
• Ability to mark a gift as belonging to a particular address for a supporter

There are two ways to handle this going forward:

• Add an address model to the database
• Simulate this through using custom fields

The first way is more correct and more reliable long term while the second is probably quicker to implement.

Inconvenient to turn a failed recurring donation, due to insufficient funds, back on

If a recurring donation card fails to charge three times, we mark it as failed so it won't be charged again. The only way via UI to reset the failure status (set n_failures back to 0) is for the donor to go to their donation management link and enter their card info again.

This is a logical requirement if the cause of the donation failures is card expiration. Cards can also fail though due to insufficient funds in the account. In that case, when the problem is corrected, it makes no sense for the donor to have to reenter credit card info.

While unclear as to how to fix this, the problem could be solved by providing a mechanism for the nonprofit to turn a recurring donation back on. Or there may be other ways?

Accessibility

The entire front end must be made fully accessible. As part of this, best practices for React should be used. The libraries to use are:

• ... List libraries ...

Associated with #5

Refunds are not showing on supporter records

Expectations:

When a user looks at a supporter records, it should include all donations and refunds to avoid confusion. Instead, this supporter is only showing donations and not related refunds.

Resolution 1: Create a display where refunds show in the Supporters display
Resolution 2: fix whatever display issue is keeping refunds from showing in Supporters display

Setup multithreading

Puma and Rails both supports multithreading. That said, it's not set up in the current code. Additionally, it's not entirely clear that Houdini's code is actually threadsafe. Setting up multithreading may require the following tasks:

• Verify that our code is threadsafe (or close to it)
• Turn on config.threadsafe! in the Rails config (since we use 3.2)
• Set up Puma multithreading (tutorial at https://devcenter.heroku.com/articles/deploying-rails-applications-with-the-puma-web-server)
• Test!

This would be best handled by someone with experience dealing with Rails and multithreading

Add proper licensing to Rails generators output

Generated code when using rails generate should have the proper license headers on top. As of now, they do not.

Two rspec tests fail when run in suite but not individually

There are two rspec tests which fail when run as part of the spec suite rake spec

Add more detail :)

Donations associated with events don't list as associated with the event on the payout report

Expectations: Donations made towards events will show up as under those events on all reporting.

Reality:
Donations made to an event (but not tickets for that event) aren't listed as associated with those events on the payout report.

Add license headers to .es6 files

During the licensing, the .es6 files were neglected to be included. Those files should be under LGPL3 or later.

Migrations are hard to merge and deal with

Currently, we have a set of migrations listed in the db. This is normally alright but becomes a huge hassle with merging a set of migrations from multiple sources. For example, if your fork is relatively far apart from master, your migrations can be quite different and ordering isn't consistent or even work correctly.

Part of the issue is that Rails migrations are a bit naive and part of it is that we need to split parts of the application up into separate parts. For example, could moving parts of the app into different Rails engines simplify this process? I'm not sure but anything we can do to simplify merging db migrations would be greatly appreciated.

Update off of Ruby 2.3 (and update to Rails)

Currently, Houdini uses Ruby 2.3 and the latest version of Rails 3.2. Ruby 2.3 is in security maintenance mode and will stop receiving security updates near the end of March, 2019. Accordingly we need to upgrade to Ruby 2.4 or higher. Additionally, as Rails 3.2 is not supported on Ruby 2.4 or higher, we need to update to at least Rails 4.2.

Add a minimal theme

On the onboard page, it makes no sense to have an in-depth header and footer. In fact, it should be very minimal so the user can focus solely on the task at hand. I'm unsure whether this feature will have use cases outside of the onboard page but who knows?

Ticket purchases via check or cash don't show proper ticket total amount in Attendees view

If you add an attendee to your event manually, via the "+ Attendee" button and select them as paying via cash or check, one would expect the total payment amount in the attendee record to be correct. Unfortunately, currently, the amount displayed is $0. I can assume the calculation calculates using only credit card transactions. This needs to be changed.

Improved Receipting for Offsite Event Tickets

Expectation:

Every supporter who buys a ticket with cash or check and is then added into the system receives a receipt that mirrors the online ticket payments.

Reality:

Receipting for event tickets logged as offsite payments do not include payment information, just general ticket information.

Resolution:

Add payment information to all event ticketing. (Most offsite donations don't include receipting, but in this case, attendees have a higher probability of wanting emailed "tickets" sent to them for their records; adding payment information standardizes receipting.)

Setup Travis CI

We want to have Travis CI working so we can do continuous integration of pull requests. This only really makes sense if #52 is completed :)

Event Ticket Feature Suggestions

The issue: Nonprofits can add discount codes to their events, but they cannot attribute those discounts to specific ticket levels. For example, a nonprofit may want to exclude VIP or sponsorship levels or only allow the discount for specific levels. They may also want to put a number cap for how many discounted tickets someone can buy (for example, they may want to limit the discount to two tickets).

Solution 1: Make it so that users can attribute or exclude discounts to specific ticket levels.
Solution 2: Allow users to place a cap on how many discounted tickets supporters can buy.

International Organization Addresses Not Supported at Account Creation

Expectations: Organizations signing up for accounts on Houdini-generated platforms outside of the US should be able to add accurate location information for their account.

The reality: When an organization creates an account, the fields that collect address information do not support addresses outside of the US.

Resolution: Create a modal that includes fields for international address information while balancing support for nonWestern address conventions and the need for security/fraud prevention.

Review and remove any CommitChange content

During the release process, it's possible some CommitChange content, either logos, wording or urls, were missed. These need to be removed and, where appropriate, possible to set via Settings.

Settings are in config/settings.yml.

Receipting for Offline Donations

Users have requested:

1. A way to print receipts (instead of emailing them) to send to donors who sent a cash or check donation and logged the donation in the system

and

2. A way to send an email receipt for offline donations (probably a button in the Supporter or Payment record).

Filename timestamps for migrations are not accurate which makes migrations a huge headache

I've noticed that the timestamp used as part of the filename for migrations are not correct. The problem is that an additional digit was accidentally added into the name. This leads to conflicts on merges because rails simply adds one to the end last digit in the timestamp portion of the migration filename.

A few things should happen:

• The timestamp in migration names should be corrected.
• The migration names in an instances DB should be corrected with a script (likely only of use to CommitChange)
• A unit test should be added to guarantee sanity of the migration filename timestamps

Add Link for Terms and Conditions and Source download

We need to add a link for terms and conditions to all pages.

This is needed due to the AGPL's requirement that a source download be available. It must have the following features:

• Include the terms of the AGPL (with webisite template exception)
• Include the any terms of the instance of Houdini, or none if none exist
• Include links to a source code download (ideally created as part of the build process)

Ideally, it should have the following features:

• an elegant way of listing the software used on the site as well as links to learn more about it.
• A link to the commit on a public github/gitlab repo so it would be easy to the user to get involved.

Edit Tags Functionality

User Expectations:

Users want the ability to edit tags without having to delete them.

The current reality:

In order to change a tag, you have to delete it and add a new tag.

This specifically affects the Mailchimp lists. As accounts work with lists in Mailchimp, it can mess up their communications workflow to have to delete a tag that has a number of people attached to it.

Fix space issue in import file names

Expectations:

The import system should be able to handle formatting differences in file uploads.

Reality: A CommitChange user tried to import a file that had a space in the name. The system said the import was complete, but no files were added. Deleting the space in the file name corrected this issue.

Resolution 1: Improve error reporting to recognize this issue.
Resolution 2: Change the import system so that a space in a file name doesn't create this error.

--
Does this still happen with the move to ActiveStorage? We should check!

Event Header/Hide Header Text Bug

Expectations: You should always be able to modify customizations to pages that you've set.

Reality: If you go to an Event and add a header image, you'll see an option in the event settings to hide the header text.

If you delete that image, the option to hide the header text (a checkbox) will disappear. However, if you have deigned to hide the header text, the option to show the header text will be gone, so you won't be able to add the header text back unless you upload another photo and go into the settings before you delete the photo.

Ticket purchases via check or cash shows as Offline Donation in the payments view.

If you add an attendee to your event manually, via the "+ Attendee" button and select them as paying via cash or check, one would expect their payment to be listed in the payments view as a Ticket Purchase. Unfortunately, right now they list as an offline donation.

NOTE: it may be relevant to look at #36 when tackling this issue.

Update Anonymous Donation System

Nonprofits expect donations, not supporters to be anonymous, and they need donations to show up in reports and displays as anonymous so that they can handle acknowledgements properly.

So far, my thoughts are:

1: Supporters should no longer be anonymous in the Supporters list, but donations should be anonymous when applicable in the display. So, it should say something like "Made a $1 anonymous donation" in the supporter entry
2. Donations should be clearly marked as anonymous in the Payments list at a glance (with icon or clear labeling)
3. Receipts should show clear labeling that a donation was anonymous for both the donor and the supporter
4. Activity feeds on profiles, campaigns, and events should show the donor as anonymous when applicable

Test Settings should be defined in spec code

Our settings system is pretty great but using .env and our test settings is a bit brittle. Ideally, the settings could be entirely defined in spec code. This allows us to more easily test different scenarios with different settings.

Support Google Maps API keys

Google is modifying their Maps API policy to prevent usage without an API key. Currently, there isn't a way to add an API key for maps used as part of Houdini.

API-ify the app

Eventually, almost all data retrieval communication should happen via an API. This API should be written as follows:

• Using Grape, Grape Entity and Grape Swagger
• Client Javascript used by the front end must be generated using the swagger-codegen.
• Swagger codegen needs to be downloaded at build time. (@wwahammy has some code for this)
• The client Javascript must be generated with Swagger Codegen before the webpack build. Should be added somewhere in the npm run watch and npm run build commands.

Additionally, the API must integrate with Rails CSRF for website usage and OAuth for remote application usage (The OAuth support can be done in a separate step)

This is a long term project and will be discussed further on our Zulip and in other meetings.

Registration page

Currently, there's no real organization registration page to speak of. Some users, like CommitChange will need this.

The registration page will have the following features:

• Have customizable theming
• Be written in React/Typescript

It should have the following features:

• Use a backend written in Grape

Add Back-end Donation Processing

User story:

Brave Trails, a nonprofit using CommitChange, finds that they have increased user error because they must use a donate button on their website or their profile page to process online donations that are called in or that have been mailed in. (This is a common scenario for many nonprofits who send out mailers.) They would like to have donate buttons that link specifically to supporters in the supporter record.

Ex. Jane Doe wants to donate $20. Jane Doe is already in the Supporter Database. To process the donation, the nonprofit would like to be able to go to Jane Doe's supporter record and add a donation through a button on her record, much like the + Donation button that already exists.

Loading overlay for React pages

One issue with React pages is that they're kind of big. Especially for people on slow connections they'll have some empty views while it's loading. We do have one already for other pages but it'll need to be modified for use on pages with React. I think for now we'll go with that particular code but I'm not sure it's exactly what we want long term.

Make it clear that Payout Reports are based on UTC time

Expectations: Have a clear display that tells users on the Payouts page or in the export itself that Payout Reports times are UTC to avoid confusion.

Reality: Some users have issues with reporting needed for tax purposes and acknowledgements, especially at the end of the year, because the time stamp isn't there.

Simplify the build process

Currently the build process in README.md is a bit cumbersome. A number of the steps could be scripted. A simple bash script, particularly for adding settings and environment variables, would be much appreciated!

Translation for parts other than Donation widget

Currently, only the donate form is translated. Eventually, the entire app should be translated.

Limit usage of polyfill

For privacy reason, we want to limit as much as possible using external websites that might track our users. One of the ones I'd like to remove is polyfill.

I don't know enough about what "advanced" features we use to know if we still need polyfill, but at least move to that version

<script>(function(d,s){
if(window.Promise&&[].includes&&Object.assign&&window.Map)return;
var sc=d.getElementsByTagName(s)[0],js=d.createElement(s);
js.src='https://cdn.polyfill.io/v2/polyfill.min.js';
sc.parentNode.insertBefore(js, sc);
}(document,'script'))</script>

This only includes polyfill.io when necessary, skipping it for modern browsers for faster load times.

Fix Unit tests :)

Currently the unit tests succeed on Eric's computer but it doesn't work in the initially cloned state. Fixing unit tests requires figuring out what is missing in the default test config and adding it.

It's possible that the test config should be static code necessary for the tests to start and then each test would be able to modify Settings as part of the test. Who knows?

Browser support policy

As a policy, CommitChange supports all IE versions 9 and higher. This has been manageable to date but isn't without difficulties. That said, this policy may not be what Houdini wants.

Streamline Mailchimp so Lists Aren't Duplicated when It's Synced Manually

Expectation: When a user clicks on a list to sync with Mailchimp, it shouldn't be duplicated in the system, new users should be added to the existing list

User Story: a CommitChange nonprofit user thought lists had to be synced by clicking on the sync button for the selected tags and was creating duplicate lists over and over again.

Resolution: Ensure that once a list is synced, it is not duplicated in the user's Mailchimp account even if a nonprofit selects that tag and submits it for another sync.

Connect transifex and github

It seems there is an option to automatically push/PR from one to the other, I need to check and install

Take saved info out of public and make that folder generated data only

In the repo, a few files are saved in public. Since this folder should be all generated info, this is a problem. These files should be moved into other folders as appropriate and then copied in via Webpack.

Additionally, after the files are moved, the public directory should be fully gitignore'd.

Improved Offsite Donation Receipting

Nonprofit User Story

Brave Trails, a CommitChange nonprofit, tries to retain email addresses for all supporters, even those who donate through check or cash. If they want to send an email receipt for donations, they have to copy the information from their payment records then either paste the information in their email client and copy the email address separately and paste into the email client to send the email, or if they're using Gmail, navigate to the Supporters dashboard, find the supporter, and send an email through the supporter entry. Both processes are time consuming and don't scale well with high volume.