packtpublishing / building-data-science-applications-with-fastapi

Building Data Science Applications with FastAPI, Published by Packt

Building Data Science Applications with FastAPI

This is the code repository for Building Data Science Applications with FastAPI, published by Packt.

Develop, manage, and deploy efficient machine learning applications with Python

What is this book about?

FastAPI is a web framework for building APIs with Python 3.6 and its later versions based on standard Python-type hints. With this book, you’ll be able to create fast and reliable data science API backends using practical examples.

This book starts with the basics of the FastAPI framework and associated modern Python programming language concepts. You'll then be taken through all the aspects of the framework, including its powerful dependency injection system and how you can use it to communicate with databases, implement authentication and integrate machine learning models. Later, you’ll cover best practices relating to testing and deployment to run a high-quality and robust application. You’ll also be introduced to the extensive ecosystem of Python data science packages. As you progress, you’ll learn how to build data science applications in Python using FastAPI. The book also demonstrates how to develop fast and efficient machine learning prediction backends and test them to achieve the best performance. Finally, you’ll see how to implement a real-time face detection system using WebSockets and a web browser as a client.

By the end of this FastAPI book, you’ll have not only learned how to implement Python in data science projects but also how to maintain and design them to meet high programming standards with the help of FastAPI.

This book covers the following exciting features:

• Explore the basics of modern Python and async I/O programming
• Get to grips with basic and advanced concepts of the FastAPI framework
• Implement a FastAPI dependency to efficiently run a machine learning model
• Integrate a simple face detection algorithm in a FastAPI backend
• Integrate common Python data science libraries in a web backend
• Deploy a performant and reliable web backend for a data science application

If you feel this book is for you, get your copy today!

Instructions and Navigations

All of the code is organized into folders. For example, Chapter2.

The code will look like the following:

from pydantic import BaseModel
class Person(BaseModel):
  first_name: str
  last_name: str
  age: int

Following is what you need for this book: This Python data science book is for data scientists and software developers interested in gaining knowledge of FastAPI and its ecosystem to build data science applications. Basic knowledge of data science and machine learning concepts and how to apply them in Python is recommended.

With the following software and hardware list you can run all code files present in the book (Chapter 1-14).

Software and Hardware List

Chapter	Software required	OS required
1 - 14	Python 3.7 and above, JavaScript	Windows, Mac OS X, and Linux (Any)

We also provide a PDF file that has color images of the screenshots/diagrams used in this book. Click here to download it.

Related products

• Hands-On Machine Learning with scikit-learn and Scientific Python Toolkits [Packt] [Amazon]

• Automated Machine Learning with AutoKeras [Packt] [Amazon]

Get to Know the Author

François Voron is graduated from the University of Saint-Étienne (France) and the University of Alicante (Spain) with a master's degree in Machine Learning and Data Mining. A full-stack web developer and a data scientist, François has a proven track record working in the SaaS industry, with a special focus on Python backends and REST API. He is also the creator and maintainer of FastAPI Users, the #1 authentication library for FastAPI, and is one of the top experts in the FastAPI community.

Download a free PDF

If you have already purchased a print or Kindle version of this book, you can get a DRM-free PDF version at no cost.
Simply click on the link to claim your free PDF.

https://packt.link/free-ebook/9781801079211

building-data-science-applications-with-fastapi's Issues

Discussion: consider merging `with` statements

Although the with statements mentioned below are written in tests, they can be merged. I am not very much sure about this but we can have a discussion on it.

1. In chapter9/chapter9_db_test.py:

29 @pytest.fixture
30 async def test_client():
31    app.dependency_overrides[get_database] = get_test_database
32    async with LifespanManager(app):
33        async with httpx.AsyncClient(app=app, base_url="http://app.io") as test_client:
34            yield test_client
35
36
37 @pytest.fixture(autouse=True, scope="module")

this can reframed as:

29 @pytest.fixture
30 async def test_client():
31    app.dependency_overrides[get_database] = get_test_database
32    async with LifespanManager(app), httpx.AsyncClient(app=app, base_url="http://app.io") as test_client:
33        yield test_client
34
35
36 @pytest.fixture(autouse=True, scope="module")

Similarly in chapter9/chapter9_app_post_test.py, chapter9/chapter9_app_test.py, chapter9/chapter9_db_test.py. There could be other collapsable with statements that could be merged that I haven't looked.

What is the use of fastapi.params?

In chapter 7, the author has written from fastapi.params import Depends
https://github.com/PacktPublishing/Building-Data-Science-Applications-with-FastAPI/blob/main/chapter7/chapter7_api_key_header.py#L2

I don't find any use in it. Can @frankie567 clarify this? Maybe, I am wrong.

Chapter 4 `PostPartialUpdate` model errors after meeting `response_model=PostPublic`

Edit

Below is no longer an issue. My confusion arose because I set exclude_unset=False to investigate its effects, then gave only {"title": "string"} when calling the endpoint. This combination leads to Internal Server Error which prompted this issue.
If we do it properly by setting exclude_unset=True, there are no issues with the question below

To allow user to call update endpoint with only title, with only content, and with none of both?

Nevertheless the investigations on effects of exclude_unset=False could be helpful to other readers.
There are 2 independent dimensions of experimentation

1. exclude_unset True/False
2. Include both title and content when calling, or have one of/both missing

---

This post is half error reporting and half what I wish was explained better.

I was testing chapter4_working_pydantic_objects_05.py.
First I edited the file to set up dummy data because it was empty and cannot be tested (will always get 404)

class DummyDatabase:
    posts: Dict[int, PostDB] = {1:PostDB(id=1,title='title1',content='content1',nb_views=10)}

Then from automated docs, I called the endpoint with {"title": "string"}, intentionally leaving out content key that was suggested by the docs (or else cannot see effect of exclude_unset=True).

Then terminal errors with

/site-packages/fastapi/routing.py", line 138, in serialize_response
    raise ValidationError(errors, field.type_)
pydantic.error_wrappers.ValidationError: 1 validation error for PostPublic
response -> content

I guess this is because PostPublic which was inherited from PostBase which had content:str type which did not allow None.

Question:
So how do we make this work? (To allow user to call update endpoint with only title, with only content, and with none of both?).
Making fields in PostBase optional doesn't seem like the correct way to handle this.

On a separate but related issue, this lesson could have been clearer on how a user could understand the effects of exclude_unset=True. To look at what this parameter is doing, and to solve the above issue, I

1. removed response_model=PostPublic from the path decorator so it doesn't constrain response types (which solves this issue, probably wrongly though)
2. Added print(updated_fields) to the code
3. called the update endpoint with {"title": "string"} (you won't see the effect of exclude_unset if you used default {"title": "string", "content": "string"} suggested by automated docs)

Response Body:

• exclude_unset=True

{
  "title": "string",
  "content": "content1",
  "id": 1,
  "nb_views": 10
}

print(updated_fields) gives {'title': 'string'}

• exclude_unset=False (default)

{
  "title": "string",
  "content": null,
  "id": 1,
  "nb_views": 10
}

print(updated_fields) gives {'title': 'string', 'content': None}

Explanation
When exclude_unset=False, the update endpoint was not given content key when called. PostPartialUpdate did not exclude what was unset/not given (content), so it filled in content with None from content: Optional[str] = None. This content:None was then put into the response through post_db.copy(update=updated_fields).
When exclude_unset=True, the update endpoint was also not given the content key when called. PostPartialUpdate excluded what was not unset/not given (content), so updated_fields contains only the title, leaving content as the original hardcoded value

excute(update_query) return influenced rows

Building-Data-Science-Applications-with-FastAPI/chapter6/sqlalchemy/app.py

Line 91 in da18675

post_id = await database.execute(update_query)

I have tried in my code.Maybe the method 'execute' for 'update' return influenced rows not the primary-key id.And I know that in the 'insert' condition new id was returned.
So maybe 'post.id' should be the parameter instead of 'post_id' in next function.

Delete irrelevant files from the project

Delete .vscode and newsgroups_model.joblib and add .vscode in .gitignore.

[Chapter 6] comment argument in create_comment() as CommentCreate instance

Hi François, first off, thanks for your amazing book! Very straight to the point in presenting the core concepts of FastAPI and its ecosystem :)

I'd just like to point out a possible typo (though quite irrelevant) in the create_comment definition in Tortoise section. If I'm not wrong, in the spirit of the previous examples the comment argument should rather be type-hinted as a CommentCreate instance.

Refering to python built-in `id` instead of `comment.post_id` in HTTPException?

I found this happening in 2 places by searching detail=f"Post in vscode.

1. https://github.com/PacktPublishing/Building-Data-Science-Applications-with-FastAPI/blob/main/chapter6/sqlalchemy_relationship/app.py#L124
2. https://github.com/PacktPublishing/Building-Data-Science-Applications-with-FastAPI/blob/main/chapter6/tortoise_relationship/app.py#L80

On the front-end, the interpolated string looks like it's printing some information about a function instead of the post id. I guess the python built-in id was used when it should be comment.post_id?

Directory problems in Chapter 6 Alembic section

Book says run alembic revision --autogenerate -m "initial migration", implicitly saying run from the folder Building-Data-Science-Applications-with-FastAPI since alembic.ini is referencing script_location = chapter6/sqlalchemy_relationship/alembic instead of the default alembic.

However this will result in errors saying FAILED: No config file 'alembic.ini' found, or file has no '[alembic]' section.

What i needed to make this work was to specify the path of alembic.ini using -c option.
alembic -c chapter6/sqlalchemy_relationship/alembic.ini revision --autogenerate -m "initial migration",
same for alembic -c chapter6/sqlalchemy_relationship/alembic.ini upgrade head.

Another issue
When I first follow along the book, i get a empty migration script

def upgrade() -> None:
    # ### commands auto generated by Alembic - please adjust! ###
    pass
    # ### end Alembic commands ###


def downgrade() -> None:
    # ### commands auto generated by Alembic - please adjust! ###
    pass
    # ### end Alembic commands ###

I found this message (https://stackoverflow.com/a/67509934/8621823) explaining that if the table is already there, the migration script will be empty at those portions, so I deleted the sqlite database, and deleted all the migration scripts, tried again and it worked.
I also had to delete the database because somewhere along the way, running alembic revision gave me ERROR [alembic.util.messaging] Can't locate revision identified by 'ca9e1b2549f3'. I'm guessing this revision was stored in the database, and it was looking for this revision under alembic/versions folder, which i had cleared.

Question

1. What is the proper way to reproduce the book's instructions and generate a non-empty migration script?
   (I wonder if my explanation of Can't locate revision error is right, and whether deleting sqlite database and all the existing migration scripts is necessary)
2. Book warns against renaming columns, what are some common workflows/scenarios with alembic assuming I don't delete the database? (doesn't feel right to delete, i did it just to reproduce the book). Would the scenario be someone adds new table/columns to a database through sqlalchemy's metadata object, which get's picked up by alembic revision command to add some new tables and columns under def upgrade in the migration script, to be applied with alembic upgrade head to change the database?

How does the way the app is run affect whether book code works?

Since chapter3_first_endpoint_01, we have been instructed to run uvicorn as uvicorn chapter3_first_endpoint_01:app, implicitly meaning we should go inside the chapter3 folder first.

Later I got stuck at chapter3_custom_response_04 because I ran it as uvicorn chapter3_custom_response_04:app, which starts uvicorn proper, but upon httpie query gives Internal Server Error.

RuntimeError: File at path /Users/hanqi/code/Building-Data-Science-Applications-with-FastAPI/chapter3/assets/cat.jpg does not exist.

To investigate:

print(__file__)
print(path.dirname(__file__))
print(path.dirname(path.dirname(__file__)))

Output:

/Users/hanqi/code/Building-Data-Science-Applications-with-FastAPI/chapter3/./chapter3_custom_response_04.py
/Users/hanqi/code/Building-Data-Science-Applications-with-FastAPI/chapter3/.
/Users/hanqi/code/Building-Data-Science-Applications-with-FastAPI/chapter3

At that moment I just hacked my way through by wrapping another path.dirname after observing the folder structure to see assets folder is on parallel level as chapter folders.
It is pretty impossible for the average reader to know that it has to be ran from outside chapter3, with uvicorn chapter3.chapter3_custom_response_04:app for the httpie GET to work, after being accustomed to all previous exercises being run from inside chapter3.

Same 3 prints when run from outside chapter3:

/Users/hanqi/code/Building-Data-Science-Applications-with-FastAPI/./chapter3/chapter3_custom_response_04.py
/Users/hanqi/code/Building-Data-Science-Applications-with-FastAPI/./chapter3
/Users/hanqi/code/Building-Data-Science-Applications-with-FastAPI/.

Question 1/2
Why is there a dot added somewhere along the path? (It caused me to have to wrap a 3rd path.dirname to move up 1 more level past the dot when I ran it wrongly).
Is that vscode/fastapi/uvicorn's behaviour? What's the logic to where in the path, or when it is added? (making a simple test.py with print(__file__) inside chapter3 folder and python -m test doesn't show this extra dot)

Only at chapter6/sqlalchemy did i realize what i did wrong in chapter3.
Again I ran uvicorn sqlalchemy.app:app (inside chapter6 folder) and got

  File "/Users/hanqi/.pyenv/versions/3.8.12/envs/lewagon/lib/python3.8/site-packages/databases/core.py", line 10, in <module>
    from sqlalchemy import text
ImportError: cannot import name 'text' from 'sqlalchemy' (/Users/hanqi/code/Building-Data-Science-Applications-with-FastAPI/chapter6/./sqlalchemy/__init__.py)

Then i saw from chapter6.sqlalchemy.models import in app.py and realized I must run it outside chapter6 with
uvicorn chapter6.sqlalchemy.app:app.

Question 2/2
From the error I see databases/core.py is trying to import something from sqlalchemy.
Is this a case of clashing names, and the sqlalchemy folder overwrote the sqlalchemy library because the folder came first in import path?
If this is true, why does running uvicorn chapter6.sqlalchemy.app:app outside chapter 6 correctly look for the sqlachemy library since the folder should still come first in import path? Is it because python does not recursively search into the Building-Data-Science-Applications-with-FastAPI folder, so doesn't know there is a sqlachemy folder there too and thus no clash this time?

P.S I tested uvicorn sqlalchemy_relationship.app:app inside chapter6 hoping there's no clash but get same error

Too much data for declared Content-Length error in DELETE post (Chapter 3)

I'm getting the following error message when testing out chapter3_response_path_parameters_02.py.

h11._util.LocalProtocolError: Too much data for declared Content-Length

This looks like a known issue. I'm guessing its because content-length: 4 in the response when returning None. Returning Response(status_code=status.HTTP_204_NO_CONTENT) seems to fix the issue (content-length zero in this case).

Chapter 9 external api test is missing the test function

chapter9_app_external_api_test.py in this code repo has the fixtures, but is missing the actual test function

From the book:

@pytest.mark.asyncio
async def test_get_employees(test_client: httpx.AsyncClient):
    response = await test_client.get("/employees")
    
    assert response.status_code == status.HTTP_200_OK
    
    json = response.json()
    assert json == MockExternalAPI.mock_data
    return response.json()

If you try to test the code as is you get a "no tests ran" message. Have to add the test function (above) to the test.py

Chapter3_headers_cookies_02.py calling from automatic docs forces user-agent entry that is ignored

The book demonstrates calling using httpie, so i wanted to test calling from the docs page since it provides convenience in

• formatting inputs
• automatically handling trailing slashes
• turns id into query parameter if forgot to add it as path parameter in chapter5_function_dependency_03

However I am unable to leave it empty. If i type gibberish, then it gives in response

{
  "user_agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7)...
}

Question
Where is the validation on the docs defined? It seems like a separate system from fastapi?
When things like Query(...,ge=0) are defined and you enter -1, the docs do not even allow Execute to be clicked, showing a red background in the textbox. What is causing this?
I also noticed in chapter5_function_dependency_02 if skip: int = Query(0, ge=0) was changed to skip: int = Query(0, gt=0), minimum: 0 will disappear from the docs, how does this happen?

What is the difference between calling from httpie, docs or other tools?

Chapter 11 ndarray.copy is not deep copy

Page 320 says "If you need to deep copy the values"
According to https://numpy.org/doc/stable/reference/generated/numpy.copy.html:

To ensure all elements within an object array are copied, use copy.deepcopy: