GithubHelp home page GithubHelp logo

rl404 / akatsuki Goto Github PK

View Code? Open in Web Editor NEW
4.0 4.0 1.0 32.23 MB

MyAnimeList anime database dump and REST API.

License: MIT License

Makefile 2.17% Dockerfile 0.22% Go 88.34% HCL 9.27%
anime database golang myanimelist rest-api

akatsuki's Introduction

Just your everyday backend developer who likes trying new things. Also has some knowledge in frontend and devops area.

Projects

  • General
  • Anime & Manga
    • Akatsuki - MyAnimeList anime database dump and REST API.
    • Akizuki - View and edit your MyAnimeList anime and manga list in a more modern design with custom tags editor.
    • Hibiki - MyAnimeList manga database dump and REST API.
    • Ikazuchi - Explore anime database with Bakemonogatari theme.
    • Inazuma - Explore manga in manga style.
    • MAL Cover - Auto generate MyAnimeList anime/manga cover.
    • MyAnimeList - MyAnimeList List Design Theme.
    • Nagato - Unofficial MyAnimeList API library for Go.
    • Sekai - Convert your MyAnimeList anime list to force-directed graph and see your anime world.
    • Verniy - Unofficial Anilist GraphQL API Library for Go.
  • Vtuber
    • Amatsukaze - Visualize vtuber data from wikia to more modern website pages.
    • Shimakaze - Vtuber wikia scraper and REST API.
  • Bot
    • Hayasui - Discord bot to get anime/manga/character/people data with interactive response message.
    • Naka - Discord bot to play youtube song.
    • Nyaa x Discord - Discord bot to notify Nyaa updates.

- rl404.com -

akatsuki's People

Contributors

dependabot[bot] avatar renovate[bot] avatar rl404 avatar

Stargazers

 avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

Forkers

gravec

akatsuki's Issues

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Detected dependencies

dockerfile
build/package/Dockerfile
  • golang 1.22
  • alpine 3.20
github-actions
.github/workflows/build.yml
  • actions/checkout v4
  • actions/setup-go v5
.github/workflows/import-terraform.yml
  • actions/checkout v4
  • google-github-actions/auth v2
  • hashicorp/setup-terraform v3
.github/workflows/publish-deploy.yml
  • actions/checkout v4
  • docker/setup-qemu-action v3
  • docker/setup-buildx-action v3
  • docker/login-action v3
  • docker/login-action v3
  • google-github-actions/auth v2
  • google-github-actions/setup-gcloud v2
  • docker/build-push-action v5
  • actions/checkout v4
  • google-github-actions/auth v2
  • hashicorp/setup-terraform v3
.github/workflows/tag.yml
  • actions/checkout v4
  • mathieudutour/github-tag-action v6.2
  • jacobtomlinson/gha-find-replace v3
  • stefanzweifel/git-auto-commit-action v5
  • mathieudutour/github-tag-action v6.2
gomod
go.mod
  • go 1.21
  • github.com/go-chi/chi/v5 v5.0.12
  • github.com/golang/protobuf v1.5.4
  • github.com/joho/godotenv v1.5.1
  • github.com/kelseyhightower/envconfig v1.4.0
  • github.com/newrelic/go-agent/v3 v3.33.0
  • github.com/rl404/fairy v0.25.2
  • github.com/rl404/nagato v0.3.4
  • github.com/spf13/cobra v1.8.1
  • github.com/swaggo/http-swagger/v2 v2.0.2
  • github.com/swaggo/swag v1.16.3
  • golang.org/x/text v0.16.0
  • google.golang.org/grpc v1.64.0
  • google.golang.org/protobuf v1.34.2
  • gorm.io/driver/mysql v1.5.7
  • gorm.io/driver/postgres v1.5.9
  • gorm.io/gorm v1.25.10
terraform
deployment/terraform/consumer.tf
deployment/terraform/cron.tf
deployment/terraform/main.tf

  • Check this box to trigger a request for Renovate to run again on this repository

M1: Cannot connect to localhost or run make docker-cron-fill

Putting some time into this again as I want to get it working on OSX.

When pulling the image from the repo there isn't an M1 version of the image. You must build it yourself. Not an issue directly but worth noting:

no matching manifest for linux/arm64/v8 in the manifest list entries

No port is exposed when running the api and consumer commands. In the read me it states:

http://localhost:45001/ is ready (port may varies depend on your .env.)

However I cannot access that and docker doesn't report any exposed ports. Having ran docker ps I don't see anything available and Code throws "No valid ports are available" when connecting through the docker extension.

I've looked through the code but I'm not a go dev and don't see where this would updated. For Mac specifically I don't think you can bind to 127.0.0.1 directly, it needs to use 0.0.0.0.

After migrating database tables, I run

make docker-cron-fill

and recieve this error:

[+] Running 1/0
 ⠿ Container akatsuki-cron-fill  Created                                                                                            0.1s
Attaching to akatsuki-cron-fill
akatsuki-cron-fill  | 2023-04-14T10:45:25Z INF config initialized
akatsuki-cron-fill  | 2023-04-14T10:45:26Z INF newrelic initialized
akatsuki-cron-fill  | 2023-04-14T10:45:26Z INF cache initialized
akatsuki-cron-fill  | 2023-04-14T10:45:26Z INF database initialized
akatsuki-cron-fill  | 2023-04-14T10:45:26Z INF pubsub initialized
akatsuki-cron-fill  | 2023-04-14T10:45:26Z INF repository anime initialized
akatsuki-cron-fill  | 2023-04-14T10:45:26Z INF repository genre initialized
akatsuki-cron-fill  | 2023-04-14T10:45:26Z INF repository studio initialized
akatsuki-cron-fill  | 2023-04-14T10:45:26Z INF repository empty id initialized
akatsuki-cron-fill  | 2023-04-14T10:45:26Z INF repository mal initialized
akatsuki-cron-fill  | 2023-04-14T10:45:26Z INF repository publisher initialized
akatsuki-cron-fill  | 2023-04-14T10:45:26Z INF service initialized
akatsuki-cron-fill  | 2023-04-14T10:45:26Z INF filling missing data...
akatsuki-cron-fill  | 2023-04-14T10:45:26Z ERR cron_fill.go:23,cron_fill.go:34,service_queue.go:72,sql.go:230 internal database error,sql.go:230 sql: Scan error on column index 0, name "max(id)": converting NULL to int64 is unsupported
akatsuki-cron-fill  | Error: internal database error
akatsuki-cron-fill  | Usage:
akatsuki-cron-fill  |   akatsuki cron fill [flags]
akatsuki-cron-fill  |
akatsuki-cron-fill  | Flags:
akatsuki-cron-fill  |   -h, --help   help for fill
akatsuki-cron-fill  |
akatsuki-cron-fill  | 2023-04-14T10:45:26Z FTL internal database error
akatsuki-cron-fill exited with code 1

I believe it's getting this error because the accepting type is int64 and the code is running MAX(id) on an empty table. I have changed this to use sql.NullInt64 or to return 0 on error but it still threw the error.

I have actually got this working by seeding the database with 1 entry to begin with using this code:

func (sql *SQL) GetMaxID(ctx context.Context) (int64, int, error) {
	var id int64
	if err := sql.db.WithContext(ctx).Model(&Anime{}).Select("COALESCE(MAX(id), 0) + 1").Row().Scan(&id); err != nil {
		return 0, http.StatusInternalServerError, errors.Wrap(ctx, errors.ErrInternalDB, err)
	}
	return id, http.StatusOK, nil
}

I then reverted it back and it worked by bringing in 30 anime at a time.

Now i have this working other than the user interface I don't understand the app I think. is this meant to scrape the whole site 30 anime at a time?

I updated the scrape cap to 10000 to see what would happen and it's running but I feel like there is something I'm missing here.

Doesn't work on M1 mac

When trying to run through the instructions on docker I get this message when trying to pull:

no matching manifest for linux/arm64/v8 in the manifest list entries

And this error when trying to build it:

make: go: No such file or directory
make: *** [clean] Error 1

Edit: This can be resolved for the build error by removing below from the docker file

GOARCH=amd64

It would be worth stating in the docs that you need to create a MAL account and set up and API app, which I discovered through looking through the env.

I also encountered this error:

akatsuki-migrate  | Error: dial tcp [::1]:3306: connect: connection refused

which can be resolved by changing the address to this, although this appears to be a general docker error:

AKATSUKI_DB_ADDRESS=docker.for.mac.localhost:3306

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.