GithubHelp home page GithubHelp logo

gonczor / black-sheep-codes Goto Github PK

View Code? Open in Web Editor NEW
4.0 1.0 1.0 562 KB

Learning platform source code

License: GNU General Public License v3.0

Shell 0.15% Dockerfile 0.40% Python 71.87% HTML 15.39% CSS 2.55% JavaScript 9.65%

black-sheep-codes's Introduction

Test StackShare

Black Sheep Learns

This contains source code for learning platform. Currently, it is in development phase.

Project currently relies on Django and Django Rest Framework with PostgreSQL as database, in the nearest future proper frontend will be implemented, preferrably using Vue.js framework.

Architecture

This is a monolithic service that exposes both frontend stuff and REST API. Frontend stuff can be found in src/frontend directory. API calls are made under /api/v1/<endpoint>. Currently they are undocumented.

Error handling

In case of non-trivial logic, i.e. when ready methods delivered by DRF or Django need to be overloaded, custom error handling is implemented. It's goal is to hide low-level details behind a generic Exceptions understood by views. Example errors I want to hide are:

  • KeyError
  • BotoCoreError
  • IntegrityError

Such errors are translated to ProcessingException and this to ProcessingApiException which inherits from APIException.

Example:

models.py:

class BaseLesson(PolymorphicModel):
    # ...
    def complete(self, user: User):
        try:
            CompletedLesson.objects.create(lesson=self, user=user)
        except IntegrityError as e:
            raise ProcessingException(detail="Already marked as complete.") from e

views.py:

class LessonViewSet(ModelViewSet):
    # ...
    @action(
        detail=True,
        methods=["PATCH", "POST"],
        url_path="mark-as-complete",
        url_name="mark_as_complete",
    )
    def mark_as_complete(self, request: Request, pk: int) -> Response:
        lesson = self.get_object()
        try:
            lesson.complete(user=self.request.user)
        except ProcessingException as e:
            raise ProcessingApiException(detail=e.detail) from e
        return Response(status=status.HTTP_204_NO_CONTENT)

Project setup

Prerequisites

You need to know Python or JS and have docker-compose installed.

Docs

Documentation can be found under localhost:8000/docs/ endpoint. Requires DEBUG to be set to be to True.

Steps

  1. Copy .env.example file to .env. It should contain all necessary variables set for local development.
  2. Run docker-compose up. By default, it will start Django's development server that you can access on http://localhost:8000/

Testing

Run docker-compose run web python manage.py test.

Deployment

Project is set up to be deployed automatically on AWS architecture after each push to the master branch.

Contributing

I am not going to pretend this isn't going to be a commercial project. While I'm very much in favor of open source, I'm also going to turn this into a commercial project. I know that some people may dislike this, so I want to make it very clear from the beginning.

Nonetheless, since this is an open source project, you may still submit Pull Requests, and I'll be happy to review them. You might go through the Issues page to search for "Good first issue" if you are looking for something simple. I'm working on it in my spare time, so I may be slow to respond, but I'm happy to share my knowledge and help those, who want to help me. If you want to learn something new - go ahead :-)

black-sheep-codes's People

Contributors

dependabot[bot] avatar gonczor avatar lucasrodrigues96 avatar sk9712 avatar

Stargazers

 avatar  avatar  avatar  avatar

Watchers

 avatar

Forkers

bangrobe

black-sheep-codes's Issues

Create endpoint for checking health

AWS Load Balancer checks health by requesting / endpoint. This serves too much data for the needs. Create an endpoint /health that will simply respond with 200 status and no data.

ValueError: Port could not be cast to integer value as '6379:6379'

View details in Rollbar: https://rollbar.com/wiktor.gonczaronek/BlackSheepLearns/items/6/

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/django/core/handlers/base.py", line 181, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/contrib/admin/options.py", line 614, in wrapper
    return self.admin_site.admin_view(view)(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/utils/decorators.py", line 130, in _wrapped_view
    response = view_func(request, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/views/decorators/cache.py", line 44, in _wrapped_view_func
    response = view_func(request, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/contrib/admin/sites.py", line 233, in inner
    return view(request, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/contrib/admin/options.py", line 1656, in change_view
    return self.changeform_view(request, object_id, form_url, extra_context)
  File "/usr/local/lib/python3.9/site-packages/django/utils/decorators.py", line 43, in _wrapper
    return bound_method(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/utils/decorators.py", line 130, in _wrapped_view
    response = view_func(request, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/contrib/admin/options.py", line 1534, in changeform_view
    return self._changeform_view(request, object_id, form_url, extra_context)
  File "/usr/local/lib/python3.9/site-packages/django/contrib/admin/options.py", line 1580, in _changeform_view
    self.save_model(request, new_object, form, not add)
  File "/app/courses/admin.py", line 14, in save_model
    test_task.apply_async()
  File "/usr/local/lib/python3.9/site-packages/celery/app/task.py", line 561, in apply_async
    return app.send_task(
  File "/usr/local/lib/python3.9/site-packages/celery/app/base.py", line 745, in send_task
    with self.producer_or_acquire(producer) as P:
  File "/usr/local/lib/python3.9/site-packages/celery/app/base.py", line 880, in producer_or_acquire
    producer, self.producer_pool.acquire, block=True,
  File "/usr/local/lib/python3.9/site-packages/celery/app/base.py", line 1264, in producer_pool
    return self.amqp.producer_pool
  File "/usr/local/lib/python3.9/site-packages/celery/app/amqp.py", line 596, in producer_pool
    self.app.connection_for_write()]
  File "/usr/local/lib/python3.9/site-packages/celery/app/base.py", line 777, in connection_for_write
    return self._connection(url or self.conf.broker_write_url, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/celery/app/base.py", line 828, in _connection
    return self.amqp.Connection(
  File "/usr/local/lib/python3.9/site-packages/kombu/connection.py", line 185, in __init__
    url_params = parse_url(hostname)
  File "/usr/local/lib/python3.9/site-packages/kombu/utils/url.py", line 44, in parse_url
    scheme, host, port, user, password, path, query = _parse_url(url)
  File "/usr/local/lib/python3.9/site-packages/kombu/utils/url.py", line 76, in url_to_parts
    parts.port,
  File "/usr/local/lib/python3.9/urllib/parse.py", line 175, in port
    raise ValueError(message) from None
ValueError: Port could not be cast to integer value as '6379:6379'Traceback (most recent call last):
  File "/usr/local/lib/python3.9/urllib/parse.py", line 172, in port
    port = int(port, 10)
ValueError: invalid literal for int() with base 10: '6379:6379'

ProgrammingError: column courses_course.description does not exist LINE 1: ...T "courses_course"."id", "courses_course"."name", "courses_c... ^

View details in Rollbar: https://rollbar.com/wiktor.gonczaronek/BlackSheepLearns/items/4/

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/django/core/handlers/base.py", line 181, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/contrib/admin/options.py", line 614, in wrapper
    return self.admin_site.admin_view(view)(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/utils/decorators.py", line 130, in _wrapped_view
    response = view_func(request, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/views/decorators/cache.py", line 44, in _wrapped_view_func
    response = view_func(request, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/contrib/admin/sites.py", line 233, in inner
    return view(request, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/utils/decorators.py", line 43, in _wrapper
    return bound_method(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/utils/decorators.py", line 130, in _wrapped_view
    response = view_func(request, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/contrib/admin/options.py", line 1811, in changelist_view
    'selection_note': _('0 of %(cnt)s selected') % {'cnt': len(cl.result_list)},
  File "/usr/local/lib/python3.9/site-packages/django/db/models/query.py", line 269, in __len__
    self._fetch_all()
  File "/usr/local/lib/python3.9/site-packages/django/db/models/query.py", line 1308, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "/usr/local/lib/python3.9/site-packages/django/db/models/query.py", line 53, in __iter__
    results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
  File "/usr/local/lib/python3.9/site-packages/django/db/models/sql/compiler.py", line 1156, in execute_sql
    cursor.execute(sql, params)
  File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 66, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/usr/local/lib/python3.9/site-packages/django/db/utils.py", line 90, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
ProgrammingError: column courses_course.description does not exist
LINE 1: ...T "courses_course"."id", "courses_course"."name", "courses_c...
                                                             ^
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
UndefinedColumn: column courses_course.description does not exist
LINE 1: ...T "courses_course"."id", "courses_course"."name", "courses_c...
                                                             ^

TemplateSyntaxError: Could not parse the remainder: '/vue.js'' from 'js/vue.js''

View details in Rollbar: https://rollbar.com/wiktor.gonczaronek/BlackSheepLearns/items/3/

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/django/core/handlers/base.py", line 204, in _get_response
    response = response.render()
  File "/usr/local/lib/python3.9/site-packages/django/template/response.py", line 105, in render
    self.content = self.rendered_content
  File "/usr/local/lib/python3.9/site-packages/django/template/response.py", line 81, in rendered_content
    template = self.resolve_template(self.template_name)
  File "/usr/local/lib/python3.9/site-packages/django/template/response.py", line 63, in resolve_template
    return select_template(template, using=self.using)
  File "/usr/local/lib/python3.9/site-packages/django/template/loader.py", line 42, in select_template
    return engine.get_template(template_name)
  File "/usr/local/lib/python3.9/site-packages/django/template/backends/django.py", line 34, in get_template
    return Template(self.engine.get_template(template_name), self)
  File "/usr/local/lib/python3.9/site-packages/django/template/engine.py", line 143, in get_template
    template, origin = self.find_template(template_name)
  File "/usr/local/lib/python3.9/site-packages/django/template/engine.py", line 125, in find_template
    template = loader.get_template(name, skip=skip)
  File "/usr/local/lib/python3.9/site-packages/django/template/loaders/base.py", line 29, in get_template
    return Template(
  File "/usr/local/lib/python3.9/site-packages/django/template/base.py", line 155, in __init__
    self.nodelist = self.compile_nodelist()
  File "/usr/local/lib/python3.9/site-packages/django/template/base.py", line 193, in compile_nodelist
    return parser.parse()
  File "/usr/local/lib/python3.9/site-packages/django/template/base.py", line 478, in parse
    raise self.error(token, e)
  File "/usr/local/lib/python3.9/site-packages/django/template/base.py", line 476, in parse
    compiled_result = compile_func(self, token)
  File "/usr/local/lib/python3.9/site-packages/django/template/loader_tags.py", line 270, in do_extends
    nodelist = parser.parse()
  File "/usr/local/lib/python3.9/site-packages/django/template/base.py", line 478, in parse
    raise self.error(token, e)
  File "/usr/local/lib/python3.9/site-packages/django/template/base.py", line 476, in parse
    compiled_result = compile_func(self, token)
  File "/usr/local/lib/python3.9/site-packages/django/template/loader_tags.py", line 213, in do_block
    nodelist = parser.parse(('endblock',))
  File "/usr/local/lib/python3.9/site-packages/django/template/base.py", line 478, in parse
    raise self.error(token, e)
  File "/usr/local/lib/python3.9/site-packages/django/template/base.py", line 476, in parse
    compiled_result = compile_func(self, token)
  File "/usr/local/lib/python3.9/site-packages/django/templatetags/static.py", line 159, in do_static
    return StaticNode.handle_token(parser, token)
  File "/usr/local/lib/python3.9/site-packages/django/templatetags/static.py", line 133, in handle_token
    path = parser.compile_filter(bits[1])
  File "/usr/local/lib/python3.9/site-packages/django/template/base.py", line 563, in compile_filter
    return FilterExpression(token, self)
  File "/usr/local/lib/python3.9/site-packages/django/template/base.py", line 662, in __init__
    raise TemplateSyntaxError("Could not parse the remainder: '%s' "
TemplateSyntaxError: Could not parse the remainder: '/vue.js'' from 'js/vue.js''

TypeError: Oops, something wrong!

View details in Rollbar: https://rollbar.com/wiktor.gonczaronek/BlackSheepLearns/items/2/

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/django/core/handlers/base.py", line 181, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
    return view_func(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/rest_framework/viewsets.py", line 125, in view
    return self.dispatch(request, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/rest_framework/views.py", line 509, in dispatch
    response = self.handle_exception(exc)
  File "/usr/local/lib/python3.9/site-packages/rest_framework/views.py", line 469, in handle_exception
    self.raise_uncaught_exception(exc)
  File "/usr/local/lib/python3.9/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
    raise exc
  File "/usr/local/lib/python3.9/site-packages/rest_framework/views.py", line 506, in dispatch
    response = handler(request, *args, **kwargs)
  File "/app/courses/views.py", line 12, in list
    raise TypeError('Oops, something wrong!')
TypeError: Oops, something wrong!

Create a secure way to use docker login

In order to raise pull limits in aws codebuild we log into the dockerhub using plaintext credentials. There should be a better way of doing this using some secrets.

NoSuchBucket: An error occurred (NoSuchBucket) when calling the PutObject operation: The specified bucket does not exist

View details in Rollbar: https://rollbar.com/wiktor.gonczaronek/BlackSheepLearns/items/5/

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/django/core/handlers/base.py", line 181, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/contrib/admin/options.py", line 614, in wrapper
    return self.admin_site.admin_view(view)(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/utils/decorators.py", line 130, in _wrapped_view
    response = view_func(request, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/views/decorators/cache.py", line 44, in _wrapped_view_func
    response = view_func(request, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/contrib/admin/sites.py", line 233, in inner
    return view(request, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/contrib/admin/options.py", line 1656, in change_view
    return self.changeform_view(request, object_id, form_url, extra_context)
  File "/usr/local/lib/python3.9/site-packages/django/utils/decorators.py", line 43, in _wrapper
    return bound_method(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/utils/decorators.py", line 130, in _wrapped_view
    response = view_func(request, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/contrib/admin/options.py", line 1534, in changeform_view
    return self._changeform_view(request, object_id, form_url, extra_context)
  File "/usr/local/lib/python3.9/site-packages/django/contrib/admin/options.py", line 1580, in _changeform_view
    self.save_model(request, new_object, form, not add)
  File "/app/courses/admin.py", line 12, in save_model
    return super().save_model(request, obj, form, change)
  File "/usr/local/lib/python3.9/site-packages/django/contrib/admin/options.py", line 1093, in save_model
    obj.save()
  File "/usr/local/lib/python3.9/site-packages/django/db/models/base.py", line 753, in save
    self.save_base(using=using, force_insert=force_insert,
  File "/usr/local/lib/python3.9/site-packages/django/db/models/base.py", line 790, in save_base
    updated = self._save_table(
  File "/usr/local/lib/python3.9/site-packages/django/db/models/base.py", line 869, in _save_table
    values = [(f, None, (getattr(self, f.attname) if raw else f.pre_save(self, False)))
  File "/usr/local/lib/python3.9/site-packages/django/db/models/base.py", line 869, in <listcomp>
    values = [(f, None, (getattr(self, f.attname) if raw else f.pre_save(self, False)))
  File "/usr/local/lib/python3.9/site-packages/django/db/models/fields/files.py", line 307, in pre_save
    file.save(file.name, file.file, save=False)
  File "/usr/local/lib/python3.9/site-packages/django/db/models/fields/files.py", line 87, in save
    self.name = self.storage.save(name, content, max_length=self.field.max_length)
  File "/usr/local/lib/python3.9/site-packages/django/core/files/storage.py", line 52, in save
    return self._save(name, content)
  File "/usr/local/lib/python3.9/site-packages/storages/backends/s3boto3.py", line 447, in _save
    obj.upload_fileobj(content, ExtraArgs=params)
  File "/usr/local/lib/python3.9/site-packages/boto3/s3/inject.py", line 619, in object_upload_fileobj
    return self.meta.client.upload_fileobj(
  File "/usr/local/lib/python3.9/site-packages/boto3/s3/inject.py", line 539, in upload_fileobj
    return future.result()
  File "/usr/local/lib/python3.9/site-packages/s3transfer/futures.py", line 106, in result
    return self._coordinator.result()
  File "/usr/local/lib/python3.9/site-packages/s3transfer/futures.py", line 265, in result
    raise self._exception
  File "/usr/local/lib/python3.9/site-packages/s3transfer/tasks.py", line 126, in __call__
    return self._execute_main(kwargs)
  File "/usr/local/lib/python3.9/site-packages/s3transfer/tasks.py", line 150, in _execute_main
    return_value = self._main(**kwargs)
  File "/usr/local/lib/python3.9/site-packages/s3transfer/upload.py", line 692, in _main
    client.put_object(Bucket=bucket, Key=key, Body=body, **extra_args)
  File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 357, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 676, in _make_api_call
    raise error_class(parsed_response, operation_name)
NoSuchBucket: An error occurred (NoSuchBucket) when calling the PutObject operation: The specified bucket does not exist

Exception: This was intended

View details in Rollbar: https://rollbar.com/wiktor.gonczaronek/BlackSheepLearns/items/7/

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/celery/app/trace.py", line 405, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/celery/app/trace.py", line 697, in __protected_call__
    return self.run(*args, **kwargs)
  File "/app/courses/tasks.py", line 56, in fail
    raise Exception("This was intended")
Exception: This was intended

Proper sign up and sign in page.

Current one is terrible. It would be nice to have a nice, modern-looking UI.

It also requires thinking about graphical layout.

Creating Lessons API

Orderable with relation to CourseSection.

Includes creating basic lesson, lesson, quiz and test variants as mentioned in #22

Create courses list pagination

Currently on /courses/ subpage we list only the first page of the results. We need to have a decent pagination for this.

List Lessons Within Course

This should appear on clicking on a course on Own courses list on main courses screen. If user has not started yet, display the first lesson, otherwise first after all completed.

Use SQS as message broker

While this may stand against #46 since Flower is not supported, it might be a good idea to use SQS instead of Redis.

  • Better auto-scaling capabilities due to the integration with cloud watch
  • Cloud watch/default monitoring can also be used for the same purposes as Flower.

Optimize queries

When retrieving course with sections and lessons, no prefetches are made and this may lead to a large number of queries executed. Measure how many are there, use prefetches and limit he number of fields retrieved.

Redirect from index to courses list

If user is already logged in, there should be a redirect from main page to courses list, i. e. / to /courses/. This can be checked by requesting /api/v1/auth/users/me/. Example response is 200 and content is:

{
    "email": "[email protected]",
    "id": 3,
    "username": "gonczor"
}

If the response is different, it means token is invalid, or expired. It should be deleted from local storage and no redirect should be performed.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.