Comments (10)
@mattstauffer This is an interesting comment on a way to get around the 5000 hour rate limit
you could have multiple tokens and switch them out depending on how many requests you have left for the hour.
from gistlog.
Also reference this if making multiple tokens possibly
https://developer.github.com/changes/2012-12-06-create-authorization-for-app/
from gistlog.
Hey @mattstauffer, what's the reason to use github's api to convert to markdown? Is it primarily the issues with the current parser?
How soon would you expect the 5k/hr limit to be a factor? Or are you hitting it already?
from gistlog.
@mattstauffer I may have gone way to simple as i did not address the api rate limit, but super simple change to use github markdown:
<?php namespace Gistlog\ContentParser;
use Github\Client;
use Github\HttpClient\Message\ResponseMediator;
class MarkdownTransformer implements Transformer
{
private $client;
public function __construct()
{
$this->client = new Client;
}
public function transform($content)
{
return $this->client->api('markdown')->render($content);
}
}
Fork here: https://github.com/quickliketurtle/gistlog/tree/github-markdown
Let me know if this is worth doing without addressing the rate limit and i'll submit a PR.
from gistlog.
@quickliketurtle The current parser has a ton of little implementation details that differ from GitHub's and fixing every little one will take a lot of work. Using GitHub's API will be much faster.
However, that means we have to hope that we don't have any more than 5000 new conversion requests per hour; otherwise we'll hit the limit and all of a sudden some page won't convert.
I doubt we'll run into that at our current traffic load, though, so that should be fine.
Your branch looks like a good start (although it looks like you removed the .env.example
file, which was probably an accident?). My biggest concern would be to make sure we're not going to run into the rate limit; can you test that it's A) caching correctly and B) authenticating with our GitHub credentials correctly. If you find it working correctly and can help verify those two things are happening correctly, I'd love for you to PR this!
from gistlog.
I started on this here as well a while ago π³https://github.com/tightenco/gistlog/commits/aw-switch-to-github-markdown
from gistlog.
Thanks. I changed it around to be more inline with what Adam was working on.
I left the original transformer and added new one for github (rather than changing the existing one). added back the .env.example file ( I renamed it to .env originally rather than copying it ).
I've got caching working ( verified by looking in the github-api-cache directory ). Created a Service Provider similar to GistClientServiceProvider.
Regarding the rate limit. It's 60 requests / hour if you don't authenticate so i'm hitting that now. ( i clicked refresh a lot ) :-)
Looking at the GistClientServiceProvider, there is a check for client_id and client_secret, however those are commented out in config/services.php and do not exist in .env.example.
Doesn't that mean the current api requests are not authenticated and should be hitting the 60 req/hr limit?
How should i test the authentication? I can use a personal access token, but for the client_id and client_secret, since secret is in the name I'm not sure you should share it. Could i register a new app on my account? What do you recommend?
from gistlog.
@mattstauffer I made a change so auth is working using just a github token. I also added a test to get the rate limit response confirming it is 5000, not 60.
I extended the github api package with a rate_limit endpoint which is what my test is using. I opened on issue on that repo asking it they would be interested in a PR for that. But until that happens maybe this test should be different...
So a recap:
- Using github api for markdown transformation.
- Response is cached.
- API Auth with token with 5000 rate limit.
Let me know your thoughts (any needed changes etc). Appreciate being able to work on this, it's fun. :-)
from gistlog.
@quickliketurtle Not sure which version has the id and secret commented out; they're working on master for sure. But token may be fine, too? Unsure of the diff, honestly.
MY DUDE writing a test for the 5000. Thank you! Been wanting a test like that for a while.
I'm curious to see what you're doing. Why don't you PR it in with your notes and I can review it and we can go from there?
from gistlog.
@mattstauffer this can be closed now
from gistlog.
Related Issues (20)
- Increase performance and caching of Gists HOT 2
- Load comments via AJAX so the initial page load isn't slow HOT 3
- Matt broke GitHub comment auth when upgrading Laravel
- Expose an RSS/Atom feed for a userβs page
- Landing page content seems to take a while to update HOT 2
- Shouldn't this be a static site? HOT 8
- Anchors aren't auto-generated for headers like in GitHub HOT 5
- Breaks at whitespace not respected in Anchor elements HOT 8
- Some small style rendering issues HOT 1
- Line height in bullet lists is too small HOT 1
- Show count of stars for a given post HOT 2
- Viewing Github Profile info below Profile Image HOT 2
- Upgrade dependencies including guzzle.
- Add Bugsnag HOT 3
- Ignore 404s on Bugsnag
- Add the count of comments to the "Comments" header, like it's shown at the top of the page
- If user isn't signed in, show the "star" button grey and pop up a signin prompt if they tap it
- Close modal when the user presses "esc" key
- Fix star/comment buttons on Safari desktop HOT 5
- Consider options for looking up and caching stargazercount that allow non-logged-in users to see them
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from gistlog.