GithubHelp home page GithubHelp logo

cybert3ch / tweeds Goto Github PK

View Code? Open in Web Editor NEW

This project forked from achyuthjoism/tweeds

0.0 0.0 0.0 277 KB

An advanced Twitter scraping & OSINT tool written in Python that doesn't use Twitter's API, allowing you to scrape a Tweets and more while evading most API limitations.

Home Page: https://dev.to/achyuthjoism/how-to-scrape-tweets-from-twitter-with-python-using-tweeds-4k2e

License: MIT License

Python 100.00%

tweeds's Introduction

Tweeds - Twitter Scraping Tool

Scrape tweets from any Twitter user profile. Twitter API alternative to scrape Twitter hashtags, threads, images, videos, statistics, and Twitter history. Export data in JSON and CSV format. This project enables you to extract large amounts of data from Twitter. It lets you do much more than the Twitter API, because it doesn't have rate limits and you don't even need to have a Twitter account, a registered app, or Twitter API key.

Do not forget to star this project.๐ŸŒŸ ๐Ÿ˜

๐Ÿ’ก Features

  • No API Key required
  • No Limit
  • No Authentication required
  • Get tweets from specific user
  • JSON and CSV export for further Data Analysis
  • And the best part it is OpenSource ๐Ÿ˜‰

โœ”๏ธ Requirements

  • Python >= 3.9

โš™ Installation

pip install tweeds

๐Ÿ’ƒ Usage

achyuthjoism-MBP ~ % tweeds

        โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ•—โ–‘โ–ˆโ–ˆโ•—โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–ˆโ–ˆโ•—โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ•—โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ•—โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ•—โ–‘โ–‘โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ•—
        โ•šโ•โ•โ–ˆโ–ˆโ•”โ•โ•โ•โ–‘โ–ˆโ–ˆโ•‘โ–‘โ–‘โ–ˆโ–ˆโ•—โ–‘โ–‘โ–ˆโ–ˆโ•‘โ–ˆโ–ˆโ•”โ•โ•โ•โ•โ•โ–ˆโ–ˆโ•”โ•โ•โ•โ•โ•โ–ˆโ–ˆโ•”โ•โ•โ–ˆโ–ˆโ•—โ–ˆโ–ˆโ•”โ•โ•โ•โ•โ•
        โ–‘โ–‘โ–‘โ–ˆโ–ˆโ•‘โ–‘โ–‘โ–‘โ–‘โ•šโ–ˆโ–ˆโ•—โ–ˆโ–ˆโ–ˆโ–ˆโ•—โ–ˆโ–ˆโ•”โ•โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ•—โ–‘โ–‘โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ•—โ–‘โ–‘โ–ˆโ–ˆโ•‘โ–‘โ–‘โ–ˆโ–ˆโ•‘โ•šโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ•—โ–‘
        โ–‘โ–‘โ–‘โ–ˆโ–ˆโ•‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–ˆโ–ˆโ–ˆโ–ˆโ•”โ•โ–ˆโ–ˆโ–ˆโ–ˆโ•‘โ–‘โ–ˆโ–ˆโ•”โ•โ•โ•โ–‘โ–‘โ–ˆโ–ˆโ•”โ•โ•โ•โ–‘โ–‘โ–ˆโ–ˆโ•‘โ–‘โ–‘โ–ˆโ–ˆโ•‘โ–‘โ•šโ•โ•โ•โ–ˆโ–ˆโ•—
        โ–‘โ–‘โ–‘โ–ˆโ–ˆโ•‘โ–‘โ–‘โ–‘โ–‘โ–‘โ•šโ–ˆโ–ˆโ•”โ•โ–‘โ•šโ–ˆโ–ˆโ•”โ•โ–‘โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ•—โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ•—โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ•”โ•โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ•”โ•
        โ–‘โ–‘โ–‘โ•šโ•โ•โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ•šโ•โ•โ–‘โ–‘โ–‘โ•šโ•โ•โ–‘โ–‘โ•šโ•โ•โ•โ•โ•โ•โ•โ•šโ•โ•โ•โ•โ•โ•โ•โ•šโ•โ•โ•โ•โ•โ•โ–‘โ•šโ•โ•โ•โ•โ•โ•โ–‘

        by Achyuth Jois M

usage: tweeds [-h] [-u U] [-s S] [--since SINCE] [--until UNTIL] [--limit LIMIT] [--near NEAR] [--geocode GEOCODE]
              [--year YEAR] [--today] [--verified] [--link LINK] [--videos] [--images] [--media] [--minLikes MINLIKES]
              [--minRetweets MINRETWEETS] [--minReplies MINREPLIES] [--json JSON] [--csv CSV] [--silent]

Scrape twitter user tweets

options:
    -h, --help            show this help message and exit
    -u U                  Users Tweets you want to scrape.
    -s S                  Search for Tweets containing this word or phrase.
    --since SINCE         Filter Tweets sent since date (Example: "2017-12-27 20:30:15" or 2017-12-27).
    --until UNTIL         Filter Tweets sent until date (Example: "2017-12-27 20:30:15" or 2017-12-27).
    --limit LIMIT         Number of Tweets to pull
    --near NEAR           Find tweets near a particular location
    --geocode GEOCODE     Search for geocoded Tweets.
    --year YEAR           Filter Tweets before specified year.
    --today               Filter Tweets from today
    --verified            Display Tweets only from verified users (Use with -s).
    --links LINK          Exclude tweets containing one or more links.
    --videos              Display only Tweets with videos.
    --images              Display only Tweets with images.
    --media               Display Tweets with only images or videos.
    --minLikes MINLIKES   Minimun likes for the tweet
    --minRetweets MINRETWEETS
                          Minimun retweets for the tweet
    --minReplies MINREPLIES
                          Minimun replies for the tweet
    --json JSON           File to write the JSON output to.
    --csv CSV             To store the output in CSV
    --silent              Dont print the tweets(Only works while taking an output!)

achyuthjoism-MBP ~ %

๐Ÿ“™ Example

CLI Example

Some simple examples to help you understand the basics:

  • tweeds -u username - Scrape all the Tweets of a user (doesn't include retweets but includes replies).
  • tweeds -u username -s pineapple - Scrape all Tweets from the user's timeline containing pineapple.
  • tweeds -s pineapple - Collect every Tweet containing pineapple from everyone's Tweets.
  • tweeds -u username --since "2015-12-20 20:30:15" - Collect Tweets that were tweeted since 2015-12-20 20:30:15.
  • tweeds -u username --since 2015-12-20 - Collect Tweets that were tweeted since 2015-12-20 00:00:00.
  • tweeds -s "Rocking Star Yash" --verified - Display Tweets by verified users that Tweeted about Rocking Star Yash.
  • tweeds --geocode "48.880048,2.385939,1km" --csv paris.csv --limit 10 - Scrape Tweets from a radius of 1km around a place in Paris and export them to a csv file.
  • tweeds -u username --images - Scrape Tweets from a user containing only images.
  • tweeds -u username --videos - Scrape Tweets from a user containing only videos.
  • tweeds -u username --media - Scarape Tweets from a user containing both images and videos.
  • tweeds -u username --links - Scrape Tweets from a user which excludes links.
  • tweeds -u username --json file.json - Scrape Tweets and save as a json file.
  • tweeds -u username --csv file.csv - Scrape Tweets and save as a csv file.

Module Example

Now Tweed can be used as a module and allows custom formatting. Example:

import tweeds

query = tweeds.Query()

query.search = "Yash Boss"
query.limit = 10
query.verified = True

tweeds.search(query)

Have fun ๐Ÿฅฐ๐Ÿ’ž

๐Ÿ“ฎ Details

Obvious disclaimer

This tool is for educational purposes only, I am not responsible for its use.

Less obvious disclaimer

This project is under MIT Licence, and you have to respect it.
Use it only in personal, criminal investigations, pentesting, or open-source projects.

tweeds's People

Contributors

achyuthjoism avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.