csharp-leaf / leaf.xnet Goto Github PK

I'm currently working on a project and am using get request to get cookies from some website and I'd like to optimize my code and increase the speed by sending head requests so I was wondering if compatibility for those could be added?

Why?

I am grabbing a value via GET which then needs to be placed inside the Cookie for the POST. How do I proceed to do this?

Hi, I need use some method http more than post and get. Is Leaf.xNet Support?

a GET request fails to get response if sent for a url that has or redirects to "https"
tested as HTTP proxy and SOCKS5 Client, both fails
tested the proxy on firefox and it worked without problems.

using Leaf.xNet;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace Poe_StashAndMtx_Dump
{
    class Program
    {
        private const string _cookieAuth = "POESESSID=0781497aaa664ae602890520936f6;";

        private static string Request(string url/*, ref CookieDictionary cookies*/)
        {
            using (var request = new HttpRequest())
            {
                HttpResponse response;
                request.Cookies.Set("POESESSID", "0781497aaa664ae60289936f83c6", ".pathofexile.com", "/");
                //request.Cookies = _cookieAuth;
                request.ConnectTimeout = 30 * 1000;
                request.ReadWriteTimeout = 30 * 1000;
                request.UserAgentRandomize();
                request.AllowAutoRedirect = false;
                request.Referer = "";

                response = request.Get(url);

                //cookies = response.Cookies;

                return response.ToString();
            }
        }
        static void Main(string[] args)
        {
            var accountName = "melirench";
            var league = "Legion";
            var getStash = Request("https://www.pathofexile.com/character-window/get-stash-items?accountName="+ accountName + "&realm=pc&league="+league+"&tabs=1&tabIndex=0&public=false");
        }
    }
}

Чем может быть вызвана проблема?

on old xnet lib
Dim GetLink As String = requets.Get("https://twitter.com/login").ToString
TextBox1.Text = requests.Cookies.ToString
and now it's not work with leaf.xnet

Great library!
But when will you provide async/await support?

Сабж.
Как можно проверить наличие куки в CookieStorage без домена?

В CookieDictionary можно было использовать ContainsKey(""), и все.

Спасибо.

Should test on luminati proxies.

xNet-Ameliorated.zip

How can i send multipart requests?
Original xNet contains AddField() and AddFile() methods.

how can i do the same things via Leaf.xNet?

I don't know if I'm not using this how it's supposed to be used or this is just playing tricks with my mind, but I'm getting a pretty weird bug with the MessageBodyLoaded property of HttpResponse objects. I got a simple code something like this:

HttpRequest httpRequest = new HttpRequest();
HttpResponse httpResponse = null;

try
{
    httpResponse = httpRequest.Get("http://www.google.com");
}
catch { }

bool? bodyLoaded = httpResponse?.MessageBodyLoaded; // This line is giving weird results

Every time I run this code bodyLoaded becomes false, even when MessageBodyLoaded is always true. Now, the weird behavior occurs by that only when stepping over that line on the debugger is that bodyLoaded becomes true. What am I missing here? I'm using the latest version (5.1.76) of Leaf.xNet from nuget.

В примерах показан такой код:

using (var request = new HttpRequest()) {
    // Do something
}
// Or
HttpRequest request = null;
try {
    request = new HttpRequest();
    // Do something 
}
catch (HttpException ex) {
    // Http error handling
    // You can use ex.Status or ex.HttpStatusCode for more details.
}
catch (Exception ex) {
	// Unhandled exceptions
}
finally {
    // Cleanup in the end if initialized
    request?.Dispose();
}

Но в таком случае куки на каждый запрос будут чистые куки, т.к request инициализируется по новой, а если эт строчку убрать, то будет утечка памяти?
Как правильно делать несколько запросов что бы сохранялись куки?

is it possible to enumerate the response cookies and get their values like the original xnet

в POST-запросе на сайт нужно передавать несколько параметров с одним именем, но разнами значениями (предполагается конвертирование в массив):
RequestParams["a[]"] = value1;
RequestParams["a[]"] = value2;
RequestParams["a[]"] = value3;
было бы неплохо добавить реализацию в базовый класс

Hey,
if a website returns a 4xx status it will throw a httpexception but i cant get the response
to find out what is wrong with the request.
Is there a solution for it and if not could you add that to the exception?

Regards.

Hello, there is again issue with cookies from domains with dot. Last working version 5.1.55.
Example of domain throwing error: Domain=.retail-mobile-prod

Can we have incapsula bypass same as CloudFlare one? Its same principles just need different cookies :)

Hello.
How can i use it with IPv6 proxies?
or can you add it, please?

thanks.

I would like to know if i can use Leaf.xNet on .net framework 4.5

When I use http proxy with Leaf.Net it say 400 error
here my code

using (var request = new Leaf.xNet.HttpRequest())
{
request.Proxy = new Leaf.xNet.HttpProxyClient("1.1.1.1", 3128);
string html = request.Get("http://api.ipify.org/").ToString();
}

But I use same http proxy with xNet lib. It by pass very good
Here is my code with xNet

using (var request = new xNet.HttpRequest())
{
request.Proxy = xNet.HttpProxyClient.Parse("1.1.1.1:3128");
string html = request.Get("http://api.ipify.org/").ToString();
> }

How can I fix to make Leaf.xNet use http proxy success?

Hello, I use the Cloudflare Service from a long time with my website. Overnight, this error happens: "Leaf.xNet.Services.Cloudflare.CloudflareException : 'Превышен лимит попыток обхода Cloudflare'".
How can I fix it?

I'm trying to send multiple Get requests in multi thread but I always get
System.NotSupportedException: 'The Write method cannot be called when another write operation is pending.'

Is there any way to get the cookie headers you get from solving captcha to keep for future requests?

I don't really familiar with github, so I will propose here. In some cases very useful function is conversion CookieStorage to/from bytes array. The additional code in CookieStorage.cs looks like:

/// <summary>
/// Сохраняет куки в массив байт.
/// </summary>
// ReSharper disable once UnusedMember.Global
public byte[] ToBytes()
{
    byte[] r;

    using (var ms = new MemoryStream())
    {
        Bf.Serialize(ms, this);
        r = ms.ToArray();
    }

    return r;
}

/// <summary>
/// Загружает <see cref="CookieStorage"/> из массива байт.
/// </summary>
/// <param name="bytes">Массив байт</param>
/// <returns>Вернет <see cref="CookieStorage"/>, который задается в свойстве <see cref="HttpRequest"/> Cookies.</returns>
// ReSharper disable once UnusedMember.Global
public static CookieStorage FromBytes(byte[] bytes)
{
    using(var ms = new MemoryStream(bytes))
        return (CookieStorage)Bf.Deserialize(ms);
}

Will appreciate if you will add this to the next release.

Hello,

I'm trying to get the request headers because I need them to use on another request, but I can't
The request headers are:

Connection: keep-alive
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
X-User-Id: 1331605
X-User-Authentication-Token: da7df6fcc9ed4df1af87890ef4a28bf7
Cache-Control: no-cache
X-Request-Id: 0dfd665c-55d2-42b6-b7be-cf9ae34da0c3
X-Runtime: 0.156734
Vary: Origin

I can see them on fiddler but I can't get them on my response.

Hello is there a way to download image or save it as base64?

Is there any exception thrown when xNet could not connect to a proxy/issue with a proxy?

I'm having some issues when dealing with custom headers for a request. So far I have something like:

httpRequest.KeepTemporaryHeadersOnRedirect = true;
httpRequest.EnableMiddleHeaders = true;

httpRequest.AddHeader("Custom-Header1", "CustomValue1");
httpRequest.AddHeader("Custom-Header2", "CustomValue2");
httpRequest.AddHeader("Custom-Header3", "CustomValue3");

while(true)
    var httpResponse = httpRequest.Post(myUrl, postData, "application/x-www-form-urlencoded; charset=UTF-8");

The issue is that only the first POST request of the loop is using the custom headers set at the beginning of the code. In order for the subsequent requests to include those custom headers as well, I need to explicitly state so every time before the request is made. I've set the KeepTemporaryHeadersOnRedirect and EnableMiddleHeaders but they don't seem to make a difference at all for my problem.
My question is how can I make those temporary headers like permanent without having to set them every single time before every request

Using Leaf.xNet, we can check if the request is a certain content type with Response.ContentType.ToString(), but how do we do this for a value in the source page?

Thanks.

Hello dear.
cloudflare updated 2 dayes ago! and aded an param is name = "s" in the challenge
please add in your bypass and fix it i waiting for your update.
tnx

Здравствуйте.
Делаю Get запрос к одному сайту и получаю такую ошибку:

Я так понимаю ошибка через то что сайт выдает какой то левый формат кукисов.
Пробую еще через свойство DontTrackCookies отключить обработку кукисов так как они мне не нужны, но тоже получаю ошибку:

Такая проблема только с этим сайтом, с другими все норм.

Hello.
How can i work with HTTP2.0 ?

Can you add possibility to work with that?

Hey, I just got a little issue with cookies retrieving. The thing is, I'm sending a GET request to this url in order to get a list of cookies before sending a further POST request and include these cookies on it. The issue is that I'm being only able to get 2 cookies out of 5. These are all the cookies I'm suppossed to get

But as seen in this other pic I'm only getting 2 out 5

Notice that I'm cancelling auto-redirections (even tho the original url doesn't have redirs), So far I don't know what else to try. Is this a bug in the lib or this is just me missing something?
Thanks in advance

when sending a POST request to a URL and it gets a redirect, either HTTP to HTTPS or WWW. to non WWW. or the reverse.
The request don't follow, and just returns empty response.

How can I force to use https? I have a page that I need to load it with https in other way it doesn't load, If I try in the browser without https doesn't work, and the error is returning me the page without the https

Hi,

I use your lib to connect with https proxy but have error. This is my code

using (var request = new HttpRequest())
{
request.UserAgent = "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.2 Safari/537.36";

            request.AddHeader("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
            request.AddHeader("Accept-Language", "en-US,en;q=0.9");

            var socksProxy = new HttpProxyClient("123.123.123.123", 443, "user", "pass");
            request.Proxy = socksProxy;

            Html = request.Get("https://whoer.net/").ToString();

}

Have error is
Unable to connect to the HTTP-server 'whoer.net'.

How can I fix it?

Hey,
I want to download a big file, but i can't do this because the content-lentgh type is Integer. I think you must pass the content-length type to long ?

Hey, first of all very good library, you've done pretty good improvements to the original xNet. Now, I dunno if I'm just being dumb here, but I'm not being able to get the response headers list. So far when I use:

EnumerateHeaders().Current;

Both Key/Value pair are just null, even tho the headers are correctly retrieved and I can see the values in the private fields in the inspector. Am I missing something?
Thanks in advance.

Hello Sir
how i can Upload File?

Im using the original xNet not yours, just want to fully make sure you get that.
Regardless, Have you worked around the weird bug with NO UID SEARCH State error when searching for messages?

This occurs right after you send the tag + UID SEARCH .... command
For some odd reason it always tends to occur with Chinese Emails like yeah.net

Hi!

I sometimes receive error "The error on the client side. Status code: 404". But in browser it's not 404 - it's absolutely 200! With IgnoreProtocolErrors this exception not raises.
So, IgnoreProtocolErrors - what means this option?

Thanks.

Hello, Are the cookies saved automatically for each domain? So if I make a new HttpRequest does it keep the cookie file from the domain?
Or each new HttpRequest has each own cookies?
I want to have Each request to have their own cookies saved.
Also how can I send json data? Is StringContent good?

Can I use headers like php CURL does as array like
Name: Value, Name: Value

Edit:
Tried to use 2threads with different HttpRequest and Different Proxy. But the only one does login normal the other does not

I would like to know if there is an english translated version..
It's quite hard to understand it in russian.

URL: http://google.com/ (without https)

var req = new HttpRequest();
req.AllowAutoRedirect = true;
req.UserAgent = Http.ChromeUserAgent();
req.Proxy = HttpProxyClient.Parse("ip:port");
var resp = req.Get("http://google.com/");
MessageBox.Show(resp);

http://prntscr.com/mlzhce

If you add https will work. This happened to other websites where there is any kind of redirects.

Any help ?

Regards

Hello, how can we post with multipart/form-data; boundary=----WebKitFormBoundary with Leaf.xNet? Is there any easy way to generate this post without any bytes or so? Or a ready to use method?

Im setting accept: application/json header and it gets overridden, it doesn't get overridden when using .Get() though.

Hello, i got this weird error. I'm trying to load a large webpage (The Content-Length is around 50 000) and when i want to get the source code using response.ToString(), this throw an error:

The weird thing of this issue, is when i use a reverse proxy like Fiddler, it loads directly, but without proxies it won't load.

Thank you in advance.