Comments (20)
Seems like you need to be logged in to an account if you want this to work in VPN / VPS.
As mentioned by @Jacodom , if you are using VPN / VPS you also need to send a cookie header, the required cookie is sessionid
, which can be obtained by logging in.
So I made a simple function to login using username and password, you can create a dummy account for this
const puppeteer = require("puppeteer");
const getSessionId = async (username: string, password: string): Promise<string> => {
return new Promise((resolve, reject) => {
(async () => {
// Open Browser and Page, then go to login page
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto("https://www.instagram.com/accounts/login/");
// Check login response
page.on("response", (response) => {
if (response.url() === "https://www.instagram.com/accounts/login/ajax/" && response.status() !== 200) reject(new Error("Failed to Login"));
});
// Create listener and listen for received cookie, `page.on("response")` doesn't show `set-cookie` header for some reason
const client = await page.target().createCDPSession();
await client.send("Network.enable");
client.on("Network.responseReceivedExtraInfo", async (response) => {
if (!response || !response.headers) return;
if (!("set-cookie" in response.headers) || !response.headers["set-cookie"].includes("sessionid")) return;
const cookie = response.headers["set-cookie"];
const sessionId = cookie.split("sessionid=")[1].split(";")[0];
await browser.close();
return resolve(sessionId);
});
// Wait for form to load, fill in username & password, then click Login button
await page.waitForSelector(".Igw0E > .-MzZI:nth-child(1) > ._9GP1n > .f0n8F > ._2hvTZ");
await page.click(".Igw0E > .-MzZI:nth-child(1) > ._9GP1n > .f0n8F > ._2hvTZ");
await page.type(".Igw0E > .-MzZI:nth-child(1) > ._9GP1n > .f0n8F > ._2hvTZ", username);
await page.type(".Igw0E > .-MzZI:nth-child(2) > ._9GP1n > .f0n8F > ._2hvTZ", password);
await page.waitForSelector("#loginForm > .Igw0E > .Igw0E > .sqdOP > .Igw0E");
await page.click("#loginForm > .Igw0E > .Igw0E > .sqdOP > .Igw0E");
})();
});
};
You can probably just use axios and send request manually, but I'm too lazy to tinker with it (it also requires some headers and password encryption, etc.), so I just used puppeteer to simulate login, just to get the sessionid
.
And when sending the request
const REQUEST_PARAMETERS = {
method: 'GET',
url: normalizePostUrl(shortcode),
headers: {
'user-agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.116 Safari/537.36',
'authority': 'www.instagram.com',
'cache-control': 'max-age=0',
'Cookie': `sessionid=${await getSessionId(instagramUsername, instagramPassword)};`
}
};
const GQL = await axios(REQUEST_PARAMETERS)
.catch(reject);
Keep in mind that if you are trying to login too many times (even though it's a correct attempt with correct username & password), Instagram might lock you from login for few minutes, so it's better to store your sessionid
in a persistent storage, probably .json
file, database, or somewhere and take the value from your persistent storage if it exists instead of sending a request to instagram everytime you need to get sessionid
.
Also sessionid
seems to be lasts for 31536000
seconds (1 year)
, so if you don't want to deal with all of the hassle above, you can just login to your Instagram account on your current PC, get the sessionid
, put it as a env
variable or something, and use that as the sessionid
cookie value when sending the request, and change it every year.
At the end, my code that I used for my Discord bot looks like this (it's in Typescript)
from user-instagram.
Yeah, on further testing and even trying scraping the HTML, it seems that Instagram has probably rate-limited these shared hosts as it's impossible to get a useful response back. So it's not an issue with this library per se, but I guess a more informative error message would be useful to implement.
from user-instagram.
Can you send me the username or post data you want to scrape ?
from user-instagram.
ofcurse:
userInstagram("boycode1") // Same as getUserData()
.then(result => {
console.log(result);
if(result.isPrivate) pageStatus = "<:lock:750353629046767728>";
else pageStatus = "<:unlock:750353629092904970>";
let rsMessaeg = new MessageEmbed()
.setAuthor(result.fullName, result.profilePic, result.link)
.setDescription("**Bio: **\n```"+result.biography+"```\n\n")
.addField("Followers", result.subscribersCount, true)
.addField("Post Counter", result.postsCount.toString(), true)
.addField("Page Status", pageStatus, true);
message.channel.send(rsMessaeg);
})
.catch(console.error);
i also tried another usernames but didn't work and i have to say it was working like for 1 hour and out of no where this error is now keep logging in console!
from user-instagram.
also, I tried your code that u posted on the intro, it has the same error.... !
from user-instagram.
I think you hit some kind of rate limit with Instagram, or the IP range owned by your VPS provider is limited, I don't really know, but the only people who sent me this error used VPS.
from user-instagram.
I Tried using on my computer and also using vpn, didn't changed for me
from user-instagram.
also, just to remind you that i checked with diffrent vps and computers......
i don't think it's kind fo rate limit or something.....
from user-instagram.
Try without the VPN, it should work !
from user-instagram.
https://www.reddit.com/r/Windscribe/comments/dc6rnr/issue_with_instagram_when_connected_to_windscribe/
I think this is an IP restriction about VPNs, the only people that reported this issue used a VPN.
from user-instagram.
it's still the same, I tried with vpn and with out that.....
from user-instagram.
This is weirdly also happening to me on Netlify when it's trying to build a Gatsby site, but works perfectly on my laptop on my home internet 🤷
3:27:36 PM: (node:3654) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'user' of undefined
3:27:36 PM: at Promise (/opt/build/repo/node_modules/user-instagram/src/scrape.js:23:37)
3:27:36 PM: at process._tickCallback (internal/process/next_tick.js:68:7)
3:27:36 PM: (node:3654) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
3:27:36 PM: (node:3654) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
from user-instagram.
The request simply does not return a valid JSON object and cannot be used by the module, weirdly this only happens to people that use VPNs or VPS
from user-instagram.
need to login when i try using node-fetch, maybe instagram making redirect or something.
from user-instagram.
I have the same issue, works well on my computer but doesn't work on my server.
Any idea where to host it? lambda?
from user-instagram.
Buy a Raspberry Pi xD
from user-instagram.
Yeah, on further testing and even trying scraping the HTML, it seems that Instagram has probably rate-limited these shared hosts as it's impossible to get a useful response back. So it's not an issue with this library per se, but I guess a more informative error message would be useful to implement.
You're right, I'll implement this asap !
from user-instagram.
Hello @EdouardCourty !
First of all thank you for this super useful library!
I was having the same issue that the other folks have been addressing with the deployable version of my site which is located in a vps server.
I tried to emulate the request that the library makes in the scrape.js file (line 14) from within the vps server with a curl command and I got no response whatsoever. Then I navigate to instagram.com with my chrome browser and I looked on if there was any other relevant header. I found a cookie header that was present in the browser but not in the scrape.js file so I added it to my curl command and finally I got a decent response.
So...A workaround could be adding this cookie header to the scrape.js file but I honestly don't believe is a sustainable solution. Also I found that the response I got has a bit different structure regarding the JSON fields.
I hope this information may be useful to address the issue.
Regards
from user-instagram.
hi
your code is very nice and clear and useability, but i get this Error
" ERROR Cannot read property 'shortcode_media' or 'user' of undefined ".
i think instagram block my isp ip, so i suggest you that add api proxy to your project.
so thanks,
from user-instagram.
Same error here. Anyone found an answer?
from user-instagram.
Related Issues (20)
- 429 response code HOT 7
- UnableToFetchPostDataError when using AWS HOT 1
- UnableToFetchUserDataError HOT 6
- Only works on local host HOT 2
- UnableToFetchUserDataError HOT 12
- Unable to get user data HOT 3
- Issue fetching the username HOT 2
- Block instagram HOT 6
- Error when deploying severless function HOT 2
- Basic use of the package HOT 9
- Typescript Support HOT 1
- does not find users HOT 6
- "Please submit an Issue on Github" HOT 3
- Safe to use?? HOT 1
- instagram.getPostData not work as expected HOT 10
- Can running getUserData too often lead to rate limiting the application? HOT 1
- No session found when running on google cloud platform HOT 2
- Error should be rejected
- Keep "The server has sent an unsuccessful response. The rate limit has been hit, retry later." Although I havent send any request HOT 2
- Unkown error HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from user-instagram.