Comments (13)
i have updated the behavior of link preview api, now the website you provided is ready for preview
from affine.
We can try to adjust the header first, and reuse user-agent
via web(extract from post-link-preview request).
curl -vvv https://www.zillow.com/homedetails/339-Buckettown-Rd-Richmond-KY-40475/341933751_zpid/ \
-H 'Accept: */*' \
-H 'Accept-Language: zh-CN,zh;q=0.9,zh-TW;q=0.8,en;q=0.7' \
-H 'Connection: keep-alive' \
-H 'DNT: 1' \
-H 'Sec-Fetch-Dest: empty' \
-H 'Sec-Fetch-Mode: cors' \
-H 'Sec-Fetch-Site: same-origin' \
-H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36' \
-H 'sec-ch-ua: "Chromium";v="122", "Not(A:Brand";v="24", "Google Chrome";v="122"' \
-H 'sec-ch-ua-mobile: ?0' \
-H 'sec-ch-ua-platform: "macOS"' | scraper 'meta'
from affine.
Issue Status: ✅ Done
✅ Done
If the issue was a bug, it’s been fixed; if it was a missing feature, it’s been implemented.
This is an automatic reply by the bot.
from affine.
@darkskygit pls take a look
from affine.
The firewall deployed by this website blocked our preview request.
We will adjust the behaviour of preview request later, but this may not have much effect.
We recommend that you contact the website to allow our preview API request.
from affine.
I'm curious to know how Notion is capable of handling this correctly. Maybe the user agent request header is being spoofed? It is not realistic to request websites to make this change.
from affine.
They used a paid service, and the paid service uses AWS machines. Since the website you mentioned is also provided by AWS, I can assume that the service may has the right to bypass AWS WAF.I'm curious to know how Notion is capable of handling this correctly. Maybe the user agent request header is being spoofed? It is not realistic to request websites to make this change.
from affine.
I know that Appflowy integrates third-party services (e.g. OpenAI), but requires the user to provide their own token to use it. Otherwise, it remains inactive. Could something similar be done for iframely? They seem to offer a good free tier that I could take advantage of.
from affine.
I know that Appflowy integrates third-party services (e.g. OpenAI), but requires the user to provide their own token to use it. Otherwise, it remains inactive. Could something similar be done for iframely? They seem to offer a good free tier that I could take advantage of.
@fundon @fourdim what do you think about this? i think integrate into worker is more insensitive, or do you think it’s better to be integrated directly by bs?
from affine.
@darkskygit
I would like to have a fallback button so that if the retrieval fails using a worker, the user can click on the fallback button to initiate a request from the client to retrieve the og meta
.
Globally, user can also set to request resources directly from client.
from affine.
@darkskygit I would like to have a fallback button so that if the retrieval fails using a worker, the user can click on the fallback button to initiate a request from the client to retrieve the . Globally, user can also set to request resources directly from client.
og meta
browser site fetch may block by cors, this strategy only be used on electron
from affine.
@darkskygit
I have tried to test the link-preview
api, and received an error.
Toggle me!
λ curl 'http://127.0.0.1:8080/api/worker/link-preview' \
-H 'Accept: */*' \
-H 'Accept-Language: zh-CN,zh;q=0.9,zh-TW;q=0.8,en;q=0.7' \
-H 'Connection: keep-alive' \
-H 'Content-Type: application/json' \
-H 'Cookie: next-auth.csrf-token=917330990c359427fdb62955adb372482773d7c462b391f6e4e4053dfe2cc8a9%7C3fb90fd8afe3d5097552c502441ac6dba2d4c9d8254a72aef5617b9befb4a119; next-auth.callback-url=http%3A%2F%2Flocalhost%3A3010' \
-H 'DNT: 1' \
-H 'Origin: http://127.0.0.1:8080' \
-H 'Referer: http://127.0.0.1:8080/workspace/nqemTGxEAjCk-knG-lCFq/sgSKARIlNxJOcnkcfVRdk' \
-H 'Sec-Fetch-Dest: empty' \
-H 'Sec-Fetch-Mode: cors' \
-H 'Sec-Fetch-Site: same-origin' \
-H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36' \
-H 'sec-ch-ua: "Chromium";v="122", "Not(A:Brand";v="24", "Google Chrome";v="122"' \
-H 'sec-ch-ua-mobile: ?0' \
-H 'sec-ch-ua-platform: "macOS"' \
--data-raw '{"url":"https://www.zillow.com/homedetails/339-Buckettown-Rd-Richmond-KY-40475/341933751_zpid/"}' -vv
* Trying 127.0.0.1:8080...
* Connected to 127.0.0.1 (127.0.0.1) port 8080
> POST /api/worker/link-preview HTTP/1.1
> Host: 127.0.0.1:8080
> Accept: */*
> Accept-Language: zh-CN,zh;q=0.9,zh-TW;q=0.8,en;q=0.7
> Connection: keep-alive
> Content-Type: application/json
> Cookie: next-auth.csrf-token=917330990c359427fdb62955adb372482773d7c462b391f6e4e4053dfe2cc8a9%7C3fb90fd8afe3d5097552c502441ac6dba2d4c9d8254a72aef5617b9befb4a119; next-auth.callback-url=http%3A%2F%2Flocalhost%3A3010
> DNT: 1
> Origin: http://127.0.0.1:8080
> Referer: http://127.0.0.1:8080/workspace/nqemTGxEAjCk-knG-lCFq/sgSKARIlNxJOcnkcfVRdk
> Sec-Fetch-Dest: empty
> Sec-Fetch-Mode: cors
> Sec-Fetch-Site: same-origin
> User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36
> sec-ch-ua: "Chromium";v="122", "Not(A:Brand";v="24", "Google Chrome";v="122"
> sec-ch-ua-mobile: ?0
> sec-ch-ua-platform: "macOS"
> Content-Length: 96
>
< HTTP/1.1 400 Bad Request
< X-Powered-By: Express
< date: Tue, 12 Mar 2024 11:23:29 GMT
< content-type: application/json;charset=UTF-8
< content-length: 24
< connection: close
< report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=bAdJY5lt6PQEZNz05q0MkbIhmU3FcG7M1kBP9x73nw5Kj5ZJkBuTZZC7VLNDfCcegoDPdezq0bw1JUO7Me0mSdkNzFdOEzQuRsi5kOc0p6Nd%2FWhjk6IbNm9AAyEcfg%3D%3D"}],"group":"cf-nel","max_age":604800}
< nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
< strict-transport-security: max-age=15552000; includeSubDomains; preload
< x-content-type-options: nosniff
< server: cloudflare
< cf-ray: 86336995099a5316-LHR
< alt-svc: h3=":443"; ma=86400
< Vary: Accept-Encoding
<
* Closing connection
{"msg":"Invalid header"}
from affine.
Is it possible to use electron backend as a proxy?
from affine.
Related Issues (20)
- Adding doc template when creating a new doc HOT 3
- Failed to upload image! crypto.subtle is undefined HOT 14
- The Calendar Day in the Journal View Doesn't Correspond to the Date in the Journal Page HOT 4
- Cannot change password on self-hosted affine installation HOT 2
- Line cannot be saved HOT 3
- Self Hosted WorkSpace Avatar HOT 5
- Urgent Security Vulnerability Discovered: Unsecured GraphQL Access HOT 3
- windows controls missing in all page route HOT 2
- Reference to other Blocks HOT 8
- Custom font families HOT 2
- Emails sent have a non-usable action button HOT 4
- Shortcut ⌥+S stands for "ś" in Polish HOT 3
- Images that support markdown syntax HOT 3
- Can't login when upgrading to beta HOT 7
- Create Collection cannot be exited with ESC HOT 1
- TypeError: Cannot read properties of undefined (reading 'history') HOT 1
- error: always show 5xx wrong HOT 4
- React ErrorBoundary TypeError: Cannot read properties of undefined (reading 'match') HOT 2
- Self-hosted: add config for turn off signup feature HOT 2
- selfhost: Upgrade to 0.13 stable causes unhealthy postgres HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from affine.