GithubHelp home page GithubHelp logo

tomp736 / pymp Goto Github PK

View Code? Open in Web Editor NEW
0.0 0.0 0.0 999 KB

proof of concept mediaplayer app using microservices

License: GNU Affero General Public License v3.0

Python 91.63% HTML 2.14% Shell 2.86% JavaScript 1.56% Smarty 1.81%

pymp's People

Contributors

tomp736 avatar

Stargazers

 avatar

Watchers

 avatar

pymp's Issues

Fffmpeg does not scale.

When processing media to generate thumbnails FfmpegService retrieves all available media. If multiple instances of pymp_server exist that are designated the role of processing media, they are not aware of who is processing what, and will all potentially (usually) process the same media.

The expected behavior is that when scaling this service, it will allow for media to be processed faster by means of splitting the work across the instances with the FfmpegService.

Collect system info and make it available ro prometheus as well as mediaregistry.

Need to collect at least free disk space and disk speed.

This information should be made available to mediaregistry-service and populated to redis.
It will be used for load balancing and health checks later on.


Example SystemInfo Class

import os
import psutil

class SystemInfo:
    def __init__(self):
        pass
    
    def get_disk_space(self, directory):
        """Returns the total available disk space in bytes on the specified directory"""
        return os.statvfs(directory).f_frsize * os.statvfs(directory).f_bavail

    def get_os_type(self):
        """Returns the OS type"""
        return os.name

    def get_memory_amount(self):
        """Returns the total amount of memory in bytes"""
        return psutil.virtual_memory().total
        
    def get_disk_read_write_speed(self):
        """Returns a tuple of disk read and write speed in bytes/sec"""
        disk_io_counters = psutil.disk_io_counters()
        return disk_io_counters.read_bytes, disk_io_counters.write_bytes

Implement watchdog for media-service.

Currently polling is used to fetch the latest index from media-service every 60 seconds. This causes issues when:

  • media-service is no longer available
  • files are added to media-service
  • files are deleted from media-service
  • files are migrated between services

Using watchdog, have the media-service notify the mediaregistry-service of changes when they occur.
Existing polling implementation can stay as a backup.


Watchdog example:

from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler

class MyHandler(FileSystemEventHandler):
    def on_created(self, event):
        if event.is_directory:
            print(f"directory created:{event.src_path}")
        else:
            print(f"file created:{event.src_path}")

    def on_deleted(self, event):
        if event.is_directory:
            print(f"directory deleted:{event.src_path}")
        else:
            print(f"file deleted:{event.src_path}")

if __name__ == "__main__":
    path = "/path/to/folder"
    observer = Observer()
    observer.schedule(MyHandler(), path, recursive=True)
    observer.start()

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.