datopian / ckan-integration-tests Goto Github PK

The same way we allow people to NPM-install this repository and run cherry-picked tests from this source base in their own customized repositories, we should allow that to happen with Mocha.

Acceptance crieteria

• It is possible to NPM install this repository elsewhere and run all our Mocha tests there
• It is possible to NPM install this repository elsewhere and run a selection of our Mocha tests there
• The docs here contains a hands-on guide on how to do that

Trying to run tests using this module as a dependency, after setting up the environment, it throws the error below.

 Error: ENOENT: no such file or directory, scandir 'cypress/fixtures'
  at Object.readdirSync (fs.js:1021:3)
  at file://test-int-tests/node_modules/ckan-integration-tests/index.js:80:10

How to reproduce:

• Set up the environment and test.js configuration.
• Install required dependencies.
• run the tests

It looks like then the cypress tries to copy the files inside the root project folder, and if it doesn't exist, throws this error.
Just create an empty project root folder called cypress\fixtures and cypress/integration and that error disappear.

Is this the expected behavior of this method?

The main intent of this library (IIRC) is to provide a core set of tests and patterns that one can reuse / build on in testing specific CKAN instances. It would be good to provide a short tutorial as to how this would be done.

After we have Mocha installed (#9) we need to port the tests from the script suggested in #8 to Mocha.

Acceptance criteria

• Running Mocha runs the create resource test
• Running Mocha runs the push blob test
• After Mocha is done, any created resource is deleted form CKAN server (provably an afterEach hook can cover that)

Following the instruction in the docs to can't run the specs.

When I run node test.js I get the following error:

The support file is missing or invalid.

Your `supportFile` is set to `/var/folders/lp/m1rj3k112s1bvqgxsy9ysm840000gp/T/ckan-uat-TBzyzH/support/index.js/index.js`, but either the file is missing or it's invalid. The `supportFile` must be a `.js` or `.coffee` file or, if you're using a preprocessor plugin, it must be supported by that plugin.

Correct your `cypress.json`, create the appropriate file, or set `supportFile` to `false` if a support file is not necessary for your project.

Or you might have renamed the extension of your `supportFile` to `.ts`. If that's the case, restart the test runner.

Learn more at https://on.cypress.io/support-file-missing-or-invalid
{ failures: 1, message: 'Could not find Cypress test run results' }

I have Cypress installed and setup with it's default folder structure.

//test.js
import cypress from "cypress";
import { CKANIntegrationTests } from "ckan-integration-tests/index.js";

const assets = new CKANIntegrationTests();
assets.addAllSpecs();

cypress
  .run(assets.options)
  .then(console.log)
  .catch(console.error)
  .finally(() => assets.cleanUp());

package.json

{
  "name": "test-int-tests",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "type": "module",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "ckan-integration-tests": "git+https://github.com/datopian/ckan-integration-tests.git",
    "ckanClient": "git+https://github.com/datopian/ckan-client-js.git#bfa869866c3bf50ada2bbc30376e0191c9c263c7",
    "cypress": "4.9.0",
    "frictionless": "git+https://github.com/frictionlessdata/frictionless-js.git#1e48a2d0c545e4294107197fa6100308ab8488e1"
  }
}

cypress.json

{
  "chromeWebSecurity": false,
  "baseUrl": "https://ckan.myserver.org/",
  "pageLoadTimeout": 120000,
  "env": {
    "FRONTEND_URL": "https://ckan.myserver.org/",
    "API_KEY": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
    "CKAN_USERNAME": "joanne.doe",
    "CKAN_PASSWORD": "joannes-very-secret-password",
    "ORG_NAME_SUFFIX": "_organization_test",
    "DATASET_NAME_SUFFIX": "_dataset_test"
  }
}

Environment

Cypress: 4.9.0
Node: v14.15.1
OS: Mac 11.0.1
Tried also with Cypress v6.0.0 but didn't work.

Current Authorization tests are not in line with the CKAN standards config values.

1. Anonymous users should access the main page without being redirect

it('Anonymous user cannot see the UI', () => {
    cy.clearCookies()
    cy.visit('/')
    cy.location().should((loc) => {
      expect(loc.pathname).to.eq('/user/login')
    })
})

2. Self-registration is enabled for anonymous users as the default value for ckan.auth.create_user_via_web is True

it('Self registration is disabled', () => {
    cy.clearCookies()
    cy.request({ url: '/user/register', failOnStatusCode: false }).then((resp) => {
      expect(resp.status).to.eq(403)
    })
})

Acceptance

• Authorization tests match CKAN default config values

Add to README docs an overview of what kinds of tooling this already provides e.g.

• What fixtures
• What existing tests

PDF previews require ckanext-pdfview to render the pdf on the browser. To keep the set of tests compatible with CKAN enterprise without any extension, these tests should be removed.

Ex.:

it('Upload PDF and check its preview', () => {
    cy.createDatasetWithoutFile().then((datasetName) => {
      cy.get('#field-image-upload').attachFile({ filePath: 'sample-pdf-with-images.pdf', fileName: 'sample-pdf-with-images.pdf' })
      cy.get('.btn-primary').click()
      cy.location('pathname').should('eq', '/dataset/' + datasetName)
      cy.get('.resource-item > .heading').click()
      // Check if PDF preview exists
      cy.get('.module-content > .nav > .active > a').contains('PDF')
      cy.get('iframe').should('exist')
      cy.deleteDataset(datasetName)
      cy.purgeDataset(datasetName)
    })
})

Acceptance

• Have any test that checks pdf previews removed

Some Cypress test specs (cypress/integration) are failing at this moment. Probably this happens because:

1. either tests make some assumptions from a CKAN environment that might not be CKAN default (e.g. #7)
2. or tests need refactor

We need to analyze each test and select those that makes sense to be here, and remove anything that maybe was inserted with a particular environment in mind. After that, we mus make sure the remaining test are green with a default CKAN installation.

We can assume as a base CKAN installation:

• CKAN 2.8 + dependencies
• ckanext-authz-service
• ckanext-external-storage
• Giftless

For some tests (non-browser based integration tests) we need to run libraries outside of Cypress (as seen on #8). A better way to handle that would be to also have Mocha setup in parallel with Cypress.

Roadmap

• Install Mocha DONE fc4e4ec
• Add before hook to create an organization DONE f8dc86f
• Addd after hook to delete the organization DONE f8dc86f
• Update the README according to new Mocha tests - @cotts
• Update the README to explain when to use Mocha and when to use Cypress (as per the issue description) - @kmanaseryan
• Create a generic env config which will be reused by Cypress and Mocha - @cotts

Acceptance

• Mocha does not leave data behind (it creates and deletes organization and resources)
• Mocha and Cypress work*
• There are instructions to run Mocha in this repository (docs)
• There are clear instructions for developers to understand when use Cypress and when use Mocha

*NOTE: Cypress still have some tests that are failing and this is not a blocker for this issue (see #12).

Reference

This is some example of what we've tried to implement using Cypress, this expresses the after and before logic intended here:

const ckanClient = require("ckanClient");
const { open } = require("frictionless.js");

const uuid = () => Math.random().toString(36).slice(2) + "_test";
const orgName = Cypress.env("ORG_NAME_SUFFIX") + uuid();
const packageName = uuid();
const headers = {
  authorization: Cypress.env("API_KEY"),
  "content-type": "application/json"
};
const client = new ckanClient.Client(
  Cypress.env("API_KEY"),
  orgName,
  packageName,
  Cypress.config().baseUrl,
  Cypress.config().baseUrl + "/_giftless"
);

describe("CKAN Client can create resource and push blob", () => {
  before(() => {
    // Create an organization
    cy.request({
      method: "POST",
      url: "api/3/action/organization_create",
      headers: headers,
      body: { name: orgName }
    });
  });

  afterEach(() => {
    // Delete the resource (if created)
    cy.request({
      method: "GET",
      url: "api/3/action/package_show?name_or_id=" + packageName,
      headers: headers
    }).then(resp => {
      cy.request({
        method: "POST",
        url: "api/3/action/package_delete",
        headers: headers,
        body: { id: resp.body.result.id }
      });
    });
  });

  after(() => {
    // Delete the organization
    cy.request({
      method: "POST",
      url: "api/3/action/organization_delete",
      headers: headers,
      body: { id: orgName }
    });
  });

  it("CKAN Client: Create resource", async () => {
    // TODO use wrap/invoke ?
    let dataset = await client.create({
      name: packageName,
      owner_org: orgName
    });
    expect(dataset.name).to.eq(packageName);
  });

  it("CKAN Client: Push blob (using async/await)", async () => {
    // first, create the resource/package
    const dataset = await client.create({
      name: packageName,
      owner_org: orgName
    });

    // then push the blob
    cy.fixture("sample.csv").then(async content => {
      const resource = open({ name: "sample.csv", data: content });
      const resp = await client.pushBlob(resource);
      expect(resp.success).to.eq(packageName);
    });
  });
});