kstateome / canvas-api

Java library for interacting with the Canvas LMS API

Canvas API Library for Java

This is a Java library which wraps the Canvas LMS REST API. It allows you to make requests to the API using Java domain objects while the low level details of the HTTP requests, authentication, etc are taken care of for you.

Some specific features include:

• Authentication using a supplied OAuth token
• Handling of API pagination
  • Ability to specify pagination page size (Canvas default is 10)
  • Optional callback procedure that gives you data chunks as they come in without having to wait for the entire request to finish
• Masquerading as other users by Canvas user ID or SIS ID
• Requesting items by Canvas ID or SIS ID

Requirements

To use this library, you must be using Java 8. We are making extensive use of Java Streams as well as the Optional class.

If you wish to contribute, you will need to be familiar with Maven to build the project. The Spring Framework is also used but only for testing.

Under Construction

This code is currently under very active development. We are implementing API calls as we need them for K-State applications so a lot of calls are still missing. However the core functionality of talking to Canvas is well established so adding individual API calls is not a difficult task. Given the number of API calls that Canvas provides and the rate at which they add more and change others, we may never achieve 100% parity on all niche features. But basic operations on accounts, users and courses should be stable over time.

Library structure

The top level object needed to execute API calls is the CanvasApiFactory class which takes a Canvas instance URL to construct. From there you can request reader and writer objects which provide the functionality to actually execute calls to the Canvas API.

API calls are grouped into reader and writer objects by the object type they handle. So any call that returns User objects or a list of User objects will be found in the UserReader class. For example, both the account level "List users in account" and the course level "List users in course" API calls are accessed through the UserReader class.

Usage

Every API call requires an OAuth token for authentication. It is up to you to acquire this token on behalf of the user. For LTI applications, this is typically done during the LTI launch process. Tokens can also be manually issued from a user's account settings page. More details on OAuth can be found in the Canvas OAuth2 documentation

Basic Query

Once you have a token, executing an API call can be as simple as creating a new CanvasApiFactory, getting a reader object from it and executing a call. A short example of retrieving an object representing the root Canvas account:

String canvasBaseUrl = "https://<institution>.instructure.com";
OauthToken oauthToken = new NonRefreshableOauthToken("mYSecreTtoKen932781");
CanvasApiFactory apiFactory = new CanvasApiFactory(canvasBaseUrl);
AccountReader acctReader = apiFactory.getReader(AccountReader.class, oauthToken);
Account rootAccount = acctReader.getSingleAccount("1").get();

API Options

Many of the API methods take some form of method-specific Options class as a parameter. This is done instead of just accepting multiple (possibly null) parameters because some API calls have a gigantic list of optional parameters. It also insulates the users of this library from compilation breaking changes just because Canvas added a new optional parameter to an API call. Information required to execute the API call is passed into the option class constructor while optional parameters are added to the option class via methods in a pseudo-builder pattern. For example the "Get Users in Course" method has one required piece of information (course ID) and 8 optional parameters. To query all students and observers who match the search term "John" you would do as follows:

GetUsersInCourseOptions options =
    new GetUsersInCourseOptions("1234") //required course ID
    .searchTerm("John")
    .enrollmentType(Arrays.asList(EnrollmentType.STUDENT, EnrollmentType.OBSERVER));
List<User> users = userReader.getUsersInCourse(options);

Masquerading

Canvas allows you to execute API calls while masquerading as another user. In order to do this, your OAuth token must have sufficient privileges. Details can be found in the Canvas documentation.

To execute a read API call while masquerading, simply add a call to the readAsCanvasUser("<canvas user ID>") or readAsSisUser("<SIS user ID>") method on the reader object. Corresponding methods also exist in writer objects

acctReader.readAsSisUser("1234567").getSingleAccount("1");

Using SIS IDs

Many Canvas objects can be queried either by their numeric Canvas ID or by the SIS ID assigned to that object during the SIS to Canvas import process. Details can be found in the Canvas documentation

To request an object by its SIS ID, prepend the ID with the appropriate string, followed by a colon. For an account:

acctReader.getSingleAccount("sis_account_id:12345");

Pagination

All API calls that return a list of objects have the potential to require pagination to get all of the requested objects. If no per_page parameter is specified on a request, Canvas defaults to only returning 10 objects at a time which can cause problems if you are trying to query large lists in a timely fashion. Details on pagination can be found in the Canvas documentation

To specify the pagination page size to be used on calls, an additional parameter is passed into the CanvasApiFactory when requesting a reader object. Note that while you can request any number you like, Canvas can also choose to ignore this number and return however many objects it wants. Currently, hosted instances of Canvas are set to max out at 100 objects per page but this is subject to change without warning. To set the pagination page size to 100:

UserReader userReader = apiFactory.getReader(UserReader.class, oauthToken, 100);

All requests made using this UserReader object will request 100 objects per page.

Pagination Callbacks

Calls that return a large list of objects can take a while even if you set the pagination page size to 100. Querying certain lists in large courses may take 15+ seconds. If you are trying to do this in real time to display to a user, you may want to take chunks of the data as they come in from Canvas and either display the partial list to the user or do some back end processing so that your results will be ready to display more quickly once the full list is done downloading. For this situation, the library provides optional callbacks for reader methods which will call a method you specify every time a page of results comes back from Canvas and pass the partial list in so you can start processing. To use this feature, it looks like this:

GetUsersInCourseOptions options = new GetUsersInCourseOptions("1146");
List<User> courseUsers = userReader.withCallback(this::processUserPage).getUsersInCourse(options);
System.out.println("Total users in course: " + courseUsers.size());

private void processUserPage(List<User> users) {
    System.out.println("Got a page of users back: " + users.size());
}

If this code is run on a course with 325 users it would print the following:

Got a page of users back: 100
Got a page of users back: 100
Got a page of users back: 100
Got a page of users back: 25
Total users in course: 325

Integration tests

To run the integration tests copy the file integration-example.properties to integration.properties and update the values to make sense for the copy of Canvas you are testing again. Be aware this does manipulate the Canvas instance so it's best used against a test/beta instance.

To run the integration tests use:

mvn -Pfailsafe verify

Making a release

This project is deployed to the central repository, once ready to release you can have the release plugin tag everything:

mvn -Prelease release:clean release:prepare

then if that completes successfully a release bundle can be pushed to the staging area of the Sonatype OSS repository with:

mvn -Prelease release:perform

We don't automatically close the staged artifacts so after checking that the files are ok you can login to the repository and release it.

Notes

Currently this library is hand coded, however there is a machine readable API specification at https://canvas.instructure.com/doc/api/api-docs.json which could be used to generate an API automatically, however it's currently using an outdated version of Swagger which doesn't work well with the existing libraries.

If you don't supply enough data to the the Canvas API on a creation request (POST) you can end up getting a 404 response. That endpoint does exist, you just haven't supplied the correct information.

License

This software is licensed under the LGPL v3 license. Please see the License.txt file in this repository for license details.

canvas-api's Issues

Models use Integers instead of Longs

Hey,

I have a set of users who are running into an interesting error when trying to communicate with their canvas institution.
com.google.gson.JsonSyntaxException: java.lang.NumberFormatException: Expected an int but was 5000000000 at line 1 column 507 path $[0].storage_quota_mb

From the looks of things in the source code a bunch of models use Integer which in Java is a 32 bit number while according to the canvas docs all of their integers are 64 bit.

I would think you would want to change basically all of your Integers in the models to Longs

Canvas api: https://canvas.instructure.com/doc/api/index.html
Right under "Schema" says "All integer ids in Canvas are 64 bit integers. String ids are also used in Canvas."

Updating course may change grade policy

We recently changed the settings on a large number of courses and did this by loading the existing settings, changing a value and saving them back, this meant we sent the whole course settings object back. When we did this it resulted in a large number of notifications going out saying that the grade policy on the course had changed (we hadn't changed it).

It appears theres a bug in Canvas that means that if the default value is false when you query the API you get the false value back, if you then set the value to false users will get a notification that the value has changed.

The workaround (until Instructure fix it) is to create a new course object with just the values that you want to update and save that course back through the API instead.

Files && List files FilesController

Please tell me how can I work from files with your library? I interest FilesController :

• Get quota informationList files;
• Get public inline preview url;
• Get file;
• Update file;
• Delete file.

I leave the link: https://canvas.instructure.com/doc/api/files.html

Request for new release version

The 1.0.24 release is nearing 18 months old, and there are a number of updates that have been committed that I would like to incorporate into my local projects.

Is it possible to request a new release that incorporates these changes?

Thanks for your consideration.

Document Reader & Writer non-thread-safety

Pretty obvious when you look at BaseImpl, but reader & writer instances are not thread safe when using responseCallback, masqueradeAs and masqueradeType. Actually hit this today while testing, figure it would be nice to at least document it for future users.

The quick fix on my end was to use proxies & ThreadLocal to cache instances.

Dealing with 5xx responses from Canvas

This is something that I ran into recently while developing a service for Canvas - it happens very, very rarely, but should I get back a 5xx exception, the API will throw an unchecked exception (CanvasException) all the way to the client.

For most normal cases, this should be okay; it is a truly exception scenario which is no fault of the API. However, the context in which this causes a problem is inside of a running ScheduledFuture which actively polls once ever X seconds (where X >= 10). I've noticed that ScheduledFutures don't like to have unhandled exceptions thrown in them, where the ultimate resolution is that the exception will ultimately be exposed once the Future times out. This poses a problem for Futures which aren't meant to time out, or time out after a very long period of time.

The way I'd have to resolve this today would be to catch an unchecked exception, which seems and feels incredibly wrong. I'm posting this more as a question to the maintainers than anything else: would it be appropriate to revise the checkHeaders method in SimpleRequestClient to not throw an exception, and then add a new method to deal with 5xx errors such that a downstream client wouldn't have to catch the exception?

Switch to a logging wrapper

As this is a low-level library it can be helpful for libraries to use a logging wrapper rather than a logging framework directly. This was when the API gets integrated with another project it can pickup the implementation that the actual project uses for logging, rather than forcing the consuming project to use log4j.

How would you feel about a PR that switches from log4j to slf4j?

There's an expanded explanation in this post: https://softwareengineering.stackexchange.com/questions/145078/should-you-log-from-library-code

CalendarApi params improvements + bugfix

Hi there!

ListCalendarEventsOptions needs to be able to set "type", currently this is not possible, therefore only events can be fetched, no assignments.

Also, i assume these items should be end_date, not start_date.

public ListCalendarEventsOptions endDate(LocalDate endDate) {
        addSingleItem("start_date", DateTimeFormatter.ISO_LOCAL_DATE.format(endDate));
        return this;
    }

    public ListCalendarEventsOptions endDate(TemporalAccessor endDate) {
        addSingleItem("start_date", DateTimeFormatter.ISO_INSTANT.format(endDate));
        return this;
    }

Assignment missing attribute

Hello:

The assignment object in canvas has an attribute that is not in the model (and curiously is not in the API documentation but it comes when you get assignments).

The attribute name is "secure_params" and it is a JWT string, signed, that contains such interesting things like the ResourceLinkId, that can be used to call the LTI AGS Lineitems endpoint to get the lineitem(s) for that assignment.

Basically the change will be adding this in the Assignment model:

private String secureParams;

    public String getSecureParams() {
        return secureParams;
    }

    public void setSecureParams(String secureParams) {
        this.secureParams = secureParams;
    }

Updating fields to null

I don't see any other issues in the repo, so let me know if there's another preferred place to discuss things.

We at UBC have a need to edit sections, including clearing the start_at and end_at properties. With the Canvas REST JSON API this is done by setting them to null. Unfortunately this conflicts with the default behaviour of GSON, which is to not serialize null fields, and GSON only let's you toggle this per GsonBuilder instance. So no field-level control.

Fortunately (for this case) a new builder is created for each request, so it would be possible to add some extra control in the API to affect it. For example:

1. Make this a class-level option. Add a new field to the CanvasObject annotation (or a whole new annotation), say includeNullValues, defaulting to false and use that when building the GsonBuilder instance. This would satisfy our use case, as we initialize the Section model instance with all existing values before applying changes and thus won't lose any data. However any existing clients that are relying on the current behaviour could start losing data.
2. New setters could be created for those fields (setStartAtToNull? clearStartAt?) and the new null behaviour only triggered if they are called. This is obviously safer but not as clean design-wise.
3. Switch to a different JSON library, like Jackson, which let's you scope null handling on a per-field basis. Would still have to do something like 1 and 2, but could limit the nulls to only fields that actually matter.
4. More complicated designs using Optional or better yet a new monad tailored for this case. This would be a lot of work for dubious gain, however.

What does everyone think? Any better ideas?

Of course the root issue is that the model classes are used for both representing values and changes to values, but solving that properly is not trivial with Java.

log4j isn't listed as a dependency

If you don't depend on log4j explicitly in the project consuming canvas-api then you get an exception when using this library:

java.lang.NoClassDefFoundError: org/apache/log4j/Logger

at edu.ksu.canvas.CanvasApiFactory.<clinit>(CanvasApiFactory.java:26)

log4j should be listed as a dependency.

Assignment Model Missing Attribute For Overrides

Hi,

Looking at the Canvas API, Assignment seems to be missing the attribute for AssignmentOverride: assignment[assignment_overrides][]

This might be very helpful to add because it seems like we can create the assignment and the override at the same time in one request rather than making two separate requests - creating an assignment before creating the override for it. We can also read in these overrides on an assignment when we request them to be included in the assignment from Canvas.

External tool not found

If I attempt to get an External Tool from Canvas using:

    ExternalToolReader reader = apiFactory.getReader(ExternalToolReader.class, token);
    Optional<ExternalTool> tool = reader.getExternalToolInAccount(accountId, toolId)

then if the tool doesn't exist in Canvas I don't get back an empty Optional but instead a edu.ksu.canvas.exception.ObjectNotFoundException is thrown.

It seems like either when the toolId doesn't exist we should either:

• get an empty Optional
• throw an exception, but not have the method return an Optional
  or is there another reason why we should have both the Optional and the exception?