zedeus / nitter Goto Github PK

Twitter.com has search, so I think Nitter should too.

FYI: You could use Searx's API (self-hostable) to implement the search (example).

Clicking on the settings icon I get following URL: https://nitter.net/settings?referer=https%3A%2F%2Ftwitter.com%2Fabout

Together with skip redirect addons this break things. For what is this referrer there?

Searching for "test" and clicking on "Tweets" on the left side is not working on nitter.snopyta.org nor nitter.42l.fr. It somehow is working on nitter.net.

Doing this search results in thousands of DNS queries to api.twitter.com.

Opening a profile with latest commit on selfhosted instance results in an exception. Full output is here: https://privatebin.net/?1c96c07ad13afaaf#DpXCg3rbkbW2yw6ysd55ys7RgchWm2sjhhtFfDNkGgAK

I am running inside a LXC.

$ nimble  -v
nimble v0.10.2 compiled at 2019-08-08 15:25:55
git hash: couldn't determine git hash

commit hash b718bbe, build with nimble build

A wiki can helpful for further documentation and also a list of public instances.
Everyone should be able to edit it, in order to enhance documentation.

Btw, probably first selfhosted (after nitter.net) instance is live: https://nitter.snopyta.org/

If you have a tweet with different replies which also have replies there is a button with x more replies. If you click it, it should show the current hidden replies.

Example: https://nitter.net/pewdiepie/status/1160983286031798272

I've found another non-working alias.
Original: https://twitter.com/F1/status/1021049105408188416/video/1
Non-working alias: https://nitter.net/F1/status/1021049105408188416/video/1
Working URL: https://nitter.net/F1/status/1021049105408188416

RSS is not dead, everyone would like to see RSS on this project ;)

Sometimes, I see twitter links ending with "/photo/1". I don't really know what it changes to usual links.

e.g: https://twitter.com/VotrubaT/status/1172509377640767489/photo/1
Nitter version: https://nitter.net/VotrubaT/status/1172509377640767489/photo/1

Nitter does not seem to recognize this kind of links and redirect to 404 error. Could you add support for it?

On Twitter I can click on someones location and search for other tweets in the same area.
Example: https://twitter.com/search?q=place%3A3ae14ea7a1ee456e

Twitter shows a date when the Tweet was published:

Nitter is missing this.

I dont know if you have a solution for geo based trends but at least the "Worldwide trends" could be shown.

I know you are currently waiting of moigagoo/norm#19 but this adds MongoDB which is probably not the best database for this kind of caching.

Why not just using PostgreSQL? If MongoDB is just for caching than okay, this will work fine. But if you add user account and following support (maybe also liking etc) I do not think MongoDB is the correct choice.

Currently there is nothing else than the user input field. It should be focused to be able to just enter a username

The search for hashtags oder regular content containing umlauts works fine, but the hyperlinks for such hashtags are broken.

as one can see in the screenshot, clicking on #Privatsphäre will search for #Privatsph:

I couldn't find a contact email for you @zedeus, so I'm just putting this here as an FYI in case it helps anyone. Please feel free to close this issue.

On iOS there's a capability called "Shortcuts":

A shortcut is a quick way to get one or more tasks done with your apps. The Shortcuts app lets you create your own shortcuts with multiple steps. - https://support.apple.com/guide/shortcuts/welcome/ios

So here's a shortcut for redirecting Twitter URLs to Nitter URLs 😄 (inspired by this Invidious shortcut):

• https://www.icloud.com/shortcuts/3e90ac68c77b45eb82cb18dab519ff76

This shortcut opens the Nitter URL in Firefox but can be configured for any browser that has its own URL Scheme, for example:

• ddgQuickLink://{url} (DuckDuckGo Browser)
• brave://open-url?url={url} (Brave Browser)

Really loving Nitter! Awesome work!

Twitter displays numbers of Likes and also displays how many photos and videos are uploaded.

https://nitter.snopyta.org/Ad_Inifinitum/status/1168670340198928385

Enabling HLS does not work with latest commit on master. It somehow works on nitter.net, do you run unpublished commits?

Sep 04 08:11:48 nitter nitter[23443]: DEBUG GET /fonts/fontello.woff2
Sep 04 08:11:48 nitter nitter[23443]: DEBUG   200 OK {"etag": @["0988e2b35116883eec2a931b7c7cfa03"], "content-type": @["font/woff2"]}
Sep 04 08:11:48 nitter nitter[23443]: DEBUG   -> public/fonts/fontello.woff2
Sep 04 08:11:48 nitter nitter[23443]: DEBUG   TCActionRaw
Sep 04 08:11:48 nitter nitter[23443]: DEBUG   200 OK {"content-type": @["video/mp4"]}
Sep 04 08:11:48 nitter nitter[23443]: DEBUG GET /favicon.ico
Sep 04 08:11:48 nitter nitter[23443]: DEBUG   404 Not Found {"content-type": @["text/html;charset=utf-8"]}
Sep 04 08:11:48 nitter nitter[23443]: DEBUG POST /enablehls
Sep 04 08:11:48 nitter nitter[23443]: DEBUG PRAGMA foreign_keys = ON
Sep 04 08:11:48 nitter nitter[23443]: DEBUG INSERT INTO prefs (hlsPlayback, mp4Playback, proxyVideos, muteVideos, autoplayGifs, hideTweetStats, hideBanner, stickyProfile, replaceYouTube, replaceTwitter) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?) <- 1, 1, 1, 0, 1, 0, 0, 1, 'invidio.us', 'nitter.snopyta.org'
Sep 04 08:11:48 nitter nitter[23443]: DEBUG   303 See Other {"set-cookie": @["preferences=9; Expires=Sat, 29 Aug 2020 08:11:47 GMT; Secure; HttpOnly; SameSite=Lax"], "location": @[""]}
Sep 04 08:11:48 nitter nitter[23443]: DEBUG GET /enablehls
Sep 04 08:11:48 nitter nitter[23443]: DEBUG PRAGMA foreign_keys = ON
Sep 04 08:11:48 nitter nitter[23443]: DEBUG SELECT id, hlsPlayback, mp4Playback, proxyVideos, muteVideos, autoplayGifs, hideTweetStats, hideBanner, stickyProfile, replaceYouTube, replaceTwitter FROM prefs WHERE id = ? <- '9'
Sep 04 08:11:48 nitter nitter[23443]: DEBUG PRAGMA foreign_keys = ON
Sep 04 08:11:48 nitter nitter[23443]: DEBUG SELECT id, username, fullname, location, website, bio, userpic, banner, following, followers, tweets, likes, media, verified, protected, joinDate, updated FROM profile WHERE lower(username) = ? <- 'enablehls'
Sep 04 08:11:48 nitter nitter[23443]: DEBUG PRAGMA foreign_keys = ON
Sep 04 08:11:48 nitter nitter[23443]: DEBUG SELECT id, username, fullname, location, website, bio, userpic, banner, following, followers, tweets, likes, media, verified, protected, joinDate, updated FROM profile WHERE lower(username) = ? <- ''
Sep 04 08:11:48 nitter nitter[23443]: DEBUG   404 Not Found {"content-type": @["text/html;charset=utf-8"]}

Tracking todos in a readme is not very nice because people who are interested in this project dont get notifications on updates.

Hi!

When an user is trying to redirect a twitter link to an unimplemented feature, she/he is redirected to a 404 error.
That would be great if, instead, users could be redirected to a special webpage for unimplemented features, saying something like "Nitter is WIP. The feature you're trying to access is not currently supported but it's on our plan. You can check what is our TODO list here."

This way, third parties would not have to take care about which URL the user is trying to redirect and then they would not have to check all implemented features and how the project is growing day after day.

Finally, that would let the user know they are not facing a bug and that humans are working on the project. (You could use this same webpage to display informations like where to submit a bug, where to talk with the community arround Nitter, where to donate, etc.)

Please add multilingual support, then we can see it in more languages 😃

This tweet https://twitter.com/ShahakShapira/status/1168900522092441600 is censored in germany

Nitter just says "Tweet not found"

DEBUG GET /ShahakShapira/status/1168900522092441600
DEBUG PRAGMA foreign_keys = ON
DEBUG SELECT id, videoId, contentId, durationMs, url, thumb, views, available, reason, playbackType FROM video WHERE videoId = ? <- ''
DEBUG   404 Not Found {"content-type": @["text/html;charset=utf-8"]}
DEBUG GET /css/style.css
DEBUG   304 Not Modified {:}
DEBUG   -> public/css/style.css
DEBUG   TCActionRaw
DEBUG GET /css/fontello.css
DEBUG   304 Not Modified {:}
DEBUG   -> public/css/fontello.css
DEBUG   TCActionRaw
DEBUG GET /logo.png
DEBUG   304 Not Modified {:}
DEBUG   -> public/logo.png
DEBUG   TCActionRaw
DEBUG GET /fonts/fontello.woff2
DEBUG   304 Not Modified {:}
DEBUG   -> public/fonts/fontello.woff2
DEBUG   TCActionRaw

Please either show some information that the tweet is censored or circumvent the censoring.

Nitter currently not support Twitter embeds. Would be nice if it does.

This project already got some attentation in the Invidious Matrix room / IRC and it would be nice if there also would be some room or platform for Nitter discussion.

Do you also use Matrix? Anything else which could be used?
If you want I can take care about all this (the room and bots etc), I also do this in Invidious Matrix room / IRC.

Currently images like banner / profile image or media is not proxied and if you wanna view it, it redirects to Twitter.

I would love to know which version of Nitter is actually being used.
Invidious provides the hash of the commit in the footer. Maybe you could do something similar in the footer or about page.

Somewhere on the nitter webpage should be a link to the github repo so visitors instantly see it is open source and free software.

SSLLabs.com give you a "B" (A is best) rating because this server does not support Forward Secrecy.
https://www.ssllabs.com/ssltest/analyze.html?d=nitter.net

Mozilla's observatory even give a "F" because of missing Content Security Policy (CSP), missing HTTP Strict Transport Security (HSTS) and missing HTTP headers like X-XSS-Protection
https://observatory.mozilla.org/analyze/nitter.net

Webbkoll confirm that: https://webbkoll.dataskydd.net/en/results?url=http%3A%2F%2Fnitter.net

Currently nitter looks on mobile the same as on desktop and you need to zoom manually otherwise you cannot read anything.

Would it be possible to make nitter more responsive and mobile friendly?

When clicking on the About button on the top right, Nitter tries to redirect me to a Twitter user called About.

You should probably move the About webpage under another path.

Nitter settings and info currently could also be an user. What if I wanna visit a user called "settings"?
I wouldnt move the users because Twitter uses twitter.com/username, what about

nitter.net/.nitter/settings or similar?

Right now nitter.net (and the whole nitter project) don't have any privacy policy, for a privacy focused project this is really needed considering how much people use privacy as a marketing argument (See OkunaOrg/okuna-www#120 for the worst example of a project that literally censor people that talk about it (they locked their issues to contributors only after I posted that, amazing for a "ethical" project)).

The search bar is focused here https://nitter.net/search?kind=custom but not here https://nitter.net/search?kind=userSearch

It would be great if we could follow the discussions by clicking on the hashtag.

In my opinion having Red text on a black (or any) background isn't really the best thing to read, you should have every text in white (or gray) while keeping red as the second color of the UI (like the line under "Tweet" for example)

Twitter bios can contain the \n character. At the moment, Nitter puts the newlines in the source. I think it should convert them to <br>

For example, mine is currently:

<div class="profile-bio"><p>Digital Troublemaker
Long haired Ⓥ geek
He/Him
&amp;: ...

It should be

<div class="profile-bio"><p>Digital Troublemaker<br>
Long haired Ⓥ geek<br>
He/Him<br>
&amp;: ...<br>

I'm not familiar with nim, but I think the issue is at https://github.com/zedeus/nitter/blob/master/src/views/profile.nim#L28 which calls linkifyText() at https://github.com/zedeus/nitter/blob/master/src/formatters.nim#L78

Or, you can add white-space: pre-wrap; to the bio. Which is how Tweets are rendered.

Add simple logfile support with a parameter.

For example ./nitter -o nitter.log

Hello,

We are hosting the nitter.42l.fr instance and we're recently having trouble with it.

Nitter randomly stops working with the "Illegal storage access" error message, indicating a segmentation fault.
Sometimes, we also encounter the "index out of bounds" error, which also forces nitter to stop.

We couldn't accurately find a correlation between the accessed pages and the crashes. It seems to be quite random, but a deeper investigation is needed.

We can't guess if those crashes are caused by a recent commit or not, since we do not have enough traffic to tell the difference.

We have some hints:

• It might be related to the crawlers wave we've recently gone through due to a misconfiguration in our robots.txt file ; a bot was crawling Twitter though our instance, generating heavy traffic.
• Due to that traffic, a lot of cache entries has been generated (250+MB). In order to remove cached data and regain disk space, we've stopped the instance, deleted cache.db and the tmp/ folder and restarted it ; we might do it again if the cache gets too big (is it possible to set a maximum cache size?).
• The Nitter instance is sending a lot of DNS queries (A? and AAAA? to api.twitter.com).
  What happens if the DNS resolvers returns an error because of too many queries at once on the same domain? (we're talking about 100+ requests per second for the same domain, some kind of random peaks when the instance is queried) Isn't nitter caching DNS queries' result ? This specific container is using the Quad9 DNS resolver.
• During our tests, we've noticed that the Nitter instance started consuming all its reserved RAM (150MB), even though it's usually consuming around 3MB. We've noticed that behavior once, but it might be related to its strange behavior.

Technical information

We are running the latest version of Nitter (latest commit in the master branch).

• The service is run within Docker 19.03.2 as a non-root user.
• The container was set as read-only (with read and write rights for its working directory only), but we've unset it in order to ensure a good test environment. The segfaults / errors are still as numerous as before, though.
• For now, the container restarts automatically when Nitter segfaults.
• The instance is running under a reverse proxy (nginx) using a similar configuration as yours.

Feel free to try crashing our instance; don't bother to ask us nginx access logs for your specified IPs if needed.

Attaching a screenshot of the container logs. We've censored the container's local IP.

Thanks for your wonderful work!

When trying to redirect https://twitter.com/search?q=chroma to https://nitter.net/search?q=chroma I don't find any result.

I guess it's because nitter endpoints are not the same as those of Twitter.

Could it be possible to use the same endpoint on Nitter? That would help a lot users to transform their links easily without having to think about "how is it handled in Nitter".

As a translator I would happily translate nitter in french (and I'm sure other people will happily translate it to other languages too) if you add support for it.

In my opinion the wasted space on the side of the content is too big, it's great for phones use but on a PC it's really too much, in my opinion you should align the left column to the left of the page and increase the size of the content, in theory it should still act the same on phones.

Hello, I've found that some status links (e.g. https://twitter.com/elonmusk/statuses/1158968745227780096) don't redirect to nitter.net correctly (https://nitter.net/elonmusk/statuses/1158968745227780096), but changed links (https://twitter.com/elonmusk/status/1158968745227780096) do (https://nitter.net/elonmusk/status/1158968745227780096).
P.S. I don't know is this a problem of Nitter by itself or just nitter.net website, so sorry if it's not a place for this thread.

I did some more testing with the RSS and the only thing missing is the fact that the author field does not contain a link to the nitter author page

Invidious implemented it like that:

    <author>
      <name>Linus Tech Tips</name>
      <uri>https://invidio.us/channel/UCXuqSBlHAE6Xw-yeJA0Tunw</uri>
    </author>

While you use:

     <dc:creator>@elonmusk</dc:creator>

According to http://www.rssboard.org/rss-profile#namespace-elements-dublin-creator there is a limitation in <dc:creator> so you need to use <author\>

Otherwise everything else is perfect and RSS appears to be feature complete

Currently you need to click on the timestamp to get the full tweet with replies. This is not very user-friendly.
If you can click anywhere on the Tweet it would be much easier and user-friendlier.

On a Twitter account you see different metadata under the bio.
There can be for example a website, a location and a join date.

I don't have a Twitter account and would like to follow RSS feeds of specific hashtags such as #esperanto that doesn't seem to show the RSS button or appending /rss to the end.

At the moment all videos start playing automatically. This can slow down the browser tab if there a lot of videos.
I would like to have a play button or autoplay only on focus (with mouse pointer).

This could also be configureable, time for settings? ;)

To ensure that the project is well tested without reading tests, repo needs to have CI and Coverage installed.

How to setup Travis for Nim

Nim coverage tool