GithubHelp home page GithubHelp logo

Comments (20)

SuspiciousLookingOwl avatar SuspiciousLookingOwl commented on June 6, 2024 3

Seems like you need to be logged in to an account if you want this to work in VPN / VPS.

As mentioned by @Jacodom , if you are using VPN / VPS you also need to send a cookie header, the required cookie is sessionid, which can be obtained by logging in.

So I made a simple function to login using username and password, you can create a dummy account for this

const puppeteer = require("puppeteer");

const getSessionId = async (username: string, password: string): Promise<string> => {
	return new Promise((resolve, reject) => {
		(async () => {
			// Open Browser and Page, then go to login page
			const browser = await puppeteer.launch();
			const page = await browser.newPage();            
            await page.goto("https://www.instagram.com/accounts/login/");
            
            // Check login response
			page.on("response", (response) => {
				if (response.url() === "https://www.instagram.com/accounts/login/ajax/" && response.status() !== 200) reject(new Error("Failed to Login"));
			});
            
			//  Create listener and listen for received cookie, `page.on("response")` doesn't show `set-cookie` header for some reason
			const client = await page.target().createCDPSession();
			await client.send("Network.enable");
			client.on("Network.responseReceivedExtraInfo", async (response) => {
				if (!response || !response.headers) return;
				if (!("set-cookie" in response.headers) || !response.headers["set-cookie"].includes("sessionid")) return;
				const cookie = response.headers["set-cookie"];
				const sessionId = cookie.split("sessionid=")[1].split(";")[0];
				await browser.close();
				return resolve(sessionId);
            });
            
			// Wait for form to load, fill in username & password, then click Login button
			await page.waitForSelector(".Igw0E > .-MzZI:nth-child(1) > ._9GP1n > .f0n8F > ._2hvTZ");
			await page.click(".Igw0E > .-MzZI:nth-child(1) > ._9GP1n > .f0n8F > ._2hvTZ");
			await page.type(".Igw0E > .-MzZI:nth-child(1) > ._9GP1n > .f0n8F > ._2hvTZ", username);
			await page.type(".Igw0E > .-MzZI:nth-child(2) > ._9GP1n > .f0n8F > ._2hvTZ", password);
			await page.waitForSelector("#loginForm > .Igw0E > .Igw0E > .sqdOP > .Igw0E");
			await page.click("#loginForm > .Igw0E > .Igw0E > .sqdOP > .Igw0E");
		})();
	});  
};

You can probably just use axios and send request manually, but I'm too lazy to tinker with it (it also requires some headers and password encryption, etc.), so I just used puppeteer to simulate login, just to get the sessionid.

And when sending the request

const REQUEST_PARAMETERS = {
           method: 'GET',
           url: normalizePostUrl(shortcode),
           headers: {
               'user-agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.116 Safari/537.36',
               'authority': 'www.instagram.com',
               'cache-control': 'max-age=0',
               'Cookie': `sessionid=${await getSessionId(instagramUsername, instagramPassword)};`
           }
       };
       const GQL = await axios(REQUEST_PARAMETERS)
         .catch(reject);

Keep in mind that if you are trying to login too many times (even though it's a correct attempt with correct username & password), Instagram might lock you from login for few minutes, so it's better to store your sessionid in a persistent storage, probably .json file, database, or somewhere and take the value from your persistent storage if it exists instead of sending a request to instagram everytime you need to get sessionid.

Also sessionid seems to be lasts for 31536000 seconds (1 year)
image, so if you don't want to deal with all of the hassle above, you can just login to your Instagram account on your current PC, get the sessionid, put it as a env variable or something, and use that as the sessionid cookie value when sending the request, and change it every year.

At the end, my code that I used for my Discord bot looks like this (it's in Typescript)

from user-instagram.

daaain avatar daaain commented on June 6, 2024 1

Yeah, on further testing and even trying scraping the HTML, it seems that Instagram has probably rate-limited these shared hosts as it's impossible to get a useful response back. So it's not an issue with this library per se, but I guess a more informative error message would be useful to implement.

from user-instagram.

EdouardCourty avatar EdouardCourty commented on June 6, 2024

Can you send me the username or post data you want to scrape ?

from user-instagram.

EhsanFox avatar EhsanFox commented on June 6, 2024

ofcurse:

userInstagram("boycode1") // Same as getUserData()
        .then(result => {
            console.log(result);
            if(result.isPrivate) pageStatus = "<:lock:750353629046767728>";
            else pageStatus = "<:unlock:750353629092904970>";
            let rsMessaeg = new MessageEmbed()
                .setAuthor(result.fullName, result.profilePic, result.link)
                .setDescription("**Bio: **\n```"+result.biography+"```\n\n")
                .addField("Followers", result.subscribersCount, true)
                .addField("Post Counter", result.postsCount.toString(), true)
                .addField("Page Status", pageStatus, true);
            message.channel.send(rsMessaeg);
        })
        .catch(console.error);

i also tried another usernames but didn't work and i have to say it was working like for 1 hour and out of no where this error is now keep logging in console!

from user-instagram.

EhsanFox avatar EhsanFox commented on June 6, 2024

also, I tried your code that u posted on the intro, it has the same error.... !

from user-instagram.

EdouardCourty avatar EdouardCourty commented on June 6, 2024

image

I think you hit some kind of rate limit with Instagram, or the IP range owned by your VPS provider is limited, I don't really know, but the only people who sent me this error used VPS.

from user-instagram.

EhsanFox avatar EhsanFox commented on June 6, 2024

I Tried using on my computer and also using vpn, didn't changed for me

from user-instagram.

EhsanFox avatar EhsanFox commented on June 6, 2024

also, just to remind you that i checked with diffrent vps and computers......

i don't think it's kind fo rate limit or something.....

from user-instagram.

EdouardCourty avatar EdouardCourty commented on June 6, 2024

Try without the VPN, it should work !

from user-instagram.

EdouardCourty avatar EdouardCourty commented on June 6, 2024

https://www.reddit.com/r/Windscribe/comments/dc6rnr/issue_with_instagram_when_connected_to_windscribe/
I think this is an IP restriction about VPNs, the only people that reported this issue used a VPN.

from user-instagram.

EhsanFox avatar EhsanFox commented on June 6, 2024

it's still the same, I tried with vpn and with out that.....

from user-instagram.

daaain avatar daaain commented on June 6, 2024

This is weirdly also happening to me on Netlify when it's trying to build a Gatsby site, but works perfectly on my laptop on my home internet 🤷

3:27:36 PM: (node:3654) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'user' of undefined
3:27:36 PM:     at Promise (/opt/build/repo/node_modules/user-instagram/src/scrape.js:23:37)
3:27:36 PM:     at process._tickCallback (internal/process/next_tick.js:68:7)
3:27:36 PM: (node:3654) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
3:27:36 PM: (node:3654) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

from user-instagram.

EdouardCourty avatar EdouardCourty commented on June 6, 2024

The request simply does not return a valid JSON object and cannot be used by the module, weirdly this only happens to people that use VPNs or VPS

from user-instagram.

arshizx avatar arshizx commented on June 6, 2024

need to login when i try using node-fetch, maybe instagram making redirect or something.

from user-instagram.

luminous8 avatar luminous8 commented on June 6, 2024

I have the same issue, works well on my computer but doesn't work on my server.
Any idea where to host it? lambda?

from user-instagram.

EdouardCourty avatar EdouardCourty commented on June 6, 2024

Buy a Raspberry Pi xD

from user-instagram.

EdouardCourty avatar EdouardCourty commented on June 6, 2024

Yeah, on further testing and even trying scraping the HTML, it seems that Instagram has probably rate-limited these shared hosts as it's impossible to get a useful response back. So it's not an issue with this library per se, but I guess a more informative error message would be useful to implement.

You're right, I'll implement this asap !

from user-instagram.

Jacodom avatar Jacodom commented on June 6, 2024

Hello @EdouardCourty !

First of all thank you for this super useful library!

I was having the same issue that the other folks have been addressing with the deployable version of my site which is located in a vps server.
I tried to emulate the request that the library makes in the scrape.js file (line 14) from within the vps server with a curl command and I got no response whatsoever. Then I navigate to instagram.com with my chrome browser and I looked on if there was any other relevant header. I found a cookie header that was present in the browser but not in the scrape.js file so I added it to my curl command and finally I got a decent response.
So...A workaround could be adding this cookie header to the scrape.js file but I honestly don't believe is a sustainable solution. Also I found that the response I got has a bit different structure regarding the JSON fields.
I hope this information may be useful to address the issue.
Regards

from user-instagram.

ali200465 avatar ali200465 commented on June 6, 2024

hi
your code is very nice and clear and useability, but i get this Error
" ERROR Cannot read property 'shortcode_media' or 'user' of undefined ".
i think instagram block my isp ip, so i suggest you that add api proxy to your project.

so thanks,

from user-instagram.

ertemishakk avatar ertemishakk commented on June 6, 2024

Same error here. Anyone found an answer?

from user-instagram.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.