matheusccouto / palpiteiro

Fantasy soccer tips with machine learning and genetic algorithm.

Matheus Couto

Activity

Portfolio

palpiteiro's Issues

Cartola Express on webapp

As a Cartola Express user
I want to request a lineup using the web app
So that I can use it on Cartola Express

Cartola FC WebApp

User story:
As a Cartola player
I want to access a website and request a line up
So that I can print it and draft my team in Cartola app

Acceptance criteria:
Be able to access a website and request a line up for the current round
Accept at least 433 schema
Accept any budget

Migrate JSON to S3

Migrate JSON files that currently sits on Azule Blob to AWS S3

Fine-tune points model

Fine-tune model with Optuna.

Draft

Penalty kick feature

Add penalty kick related features

Data quality on training set

The query above produced outliers that should be checked:

• total_points_repr_last_5
• total_points_last_5
• total_points_club_last_5
• total_points
• total_allowed_points_opponent_last_5
• prob_tie
• offensive_points_repr_last_5
• offensive_points_last_5
• offensive_points_club_last_5
• offensive_allowed_points_opponent_last_5
• defensive_points_repr_last_5

The query above produced excess zeros that should be checked:

• total_points_repr_last_5
• offensive_points_repr_last_5
• importance_opponent
• importance_club
• defensive_points_repr_last_5
• defensive_points_last_5
• avg_odds_opponent
• avg_odds_draw
• avg_odds_club

SELECT
    datetime(timestamp) AS timestamp,
    position,
    total_points_last_5,
    offensive_points_last_5,
    defensive_points_last_5,
    total_points_repr_last_5,
    offensive_points_repr_last_5,
    defensive_points_repr_last_5,
    spi_club,
    spi_opponent,
    prob_club,
    prob_opponent,
    prob_tie,
    importance_club,
    importance_opponent,
    proj_score_club,
    proj_score_opponent,
    total_points_club_last_5,
    offensive_points_club_last_5,
    defensive_points_club_last_5,
    total_allowed_points_opponent_last_5,
    offensive_allowed_points_opponent_last_5,
    defensive_allowed_points_opponent_last_5,
    penalties_club_last_5,
    penalties_opponent_last_5,
    received_penalties_club_last_5,
    received_penalties_opponent_last_5,
    played_last_5,
    avg_odds_club,
    avg_odds_opponent,
    avg_odds_draw,
    IF(total_points <= 0.01, 0.01, total_points) AS total_points
FROM
    palpiteiro.fct_player
WHERE
    played IS TRUE
    AND played_last_5_at > 0
    AND position != 'coach'

CartolaFC Backend

Health features

Include features based on injured players

Participation predictions

As a user
I want to have higher confidence if the player is expected to play
So I do not lose points because a player did not participate

Establish MLOps

Establish MLOps for the points model

Cartola FC scraper

• Create tables
• Develop extract players function
• Develop extract matches function
• Develop transform players function
• Develop transform matches function
• Develop load function
• Define pipeline
• Set alarms

Migrate cartola.atletas to CockroachDB

Migrate SQL database that currently sits on Azure SQL to CockroachDB

Telegram Bot

As a Cartola user,
I want to request my lineup via Telegram
So I do not have to open the browser and type an address

Transfermarkt scraper

Scrape data from transfermarkt

Cartola Express scraper

Scrape data from Cartola Express

• Create tables
• Develop extract function
• Develop transform function
• Define pipeline
• Set alarms

Forecast

FiveThirtyEight scraper

• Create tables
• Develop extract function
• Define pipeline
• Set alarms

Cartola Bot

As a Cartola user,
I want a bot to submit the lineup for myself,
So that I do not have to do it myself

Features associations

Those features are highly associated and maybe some should be dropped.

"avg_odds_club" & "avg_odds_opponent" +0.701
"penalties_opponent_last_5" & "received_penalties_club_last_5" +0.658
"prob_club" & "prob_opponent" +0.652
"penalties_club_last_5" &
"received_penalties_opponent_last_5" +0.635 "prob_opponent" &
"proj_score_opponent" +0.411
"prob_club" & "proj_score_club"+0.407
"offensive_points_last_5" & "offensive_points_repr_last_5"+0.346
"avg_odds_opponent" & "prob_opponent" +0.334
"avg_odds_club" & "prob_club" +0.333
"avg_odds_opponent" & "prob_club"+0.331

Odds Analytics

Develop and document Odds models

Scrape odds

Fix 2017 scouts

Missing scouts on at least 2017 round 1 and 2

Missing values on training set

The query above produced 88 missing values on:

• total_points_club_last_5
• offensive_points_club_last_5
• defensive_points_club_last_5
• total_allowed_points_opponent_last_5
• offensive_allowed...s_opponent_last_5
• defensive_allowed...s_opponent_last_5

SELECT
    datetime(timestamp) AS timestamp,
    position,
    total_points_last_5,
    offensive_points_last_5,
    defensive_points_last_5,
    total_points_repr_last_5,
    offensive_points_repr_last_5,
    defensive_points_repr_last_5,
    spi_club,
    spi_opponent,
    prob_club,
    prob_opponent,
    prob_tie,
    importance_club,
    importance_opponent,
    proj_score_club,
    proj_score_opponent,
    total_points_club_last_5,
    offensive_points_club_last_5,
    defensive_points_club_last_5,
    total_allowed_points_opponent_last_5,
    offensive_allowed_points_opponent_last_5,
    defensive_allowed_points_opponent_last_5,
    penalties_club_last_5,
    penalties_opponent_last_5,
    received_penalties_club_last_5,
    received_penalties_opponent_last_5,
    played_last_5,
    avg_odds_club,
    avg_odds_opponent,
    avg_odds_draw,
    IF(total_points <= 0.01, 0.01, total_points) AS total_points
FROM
    palpiteiro.fct_player
WHERE
    played IS TRUE
    AND played_last_5_at > 0
    AND position != 'coach'

Update docs

Documentation is outdated and should be updated

Rei do Pitaco scraper

Scrape Rei do Pitaco data

• Create tables
• Create extract function
• Create transform function
• Define pipeline
• Set alarms