By Evangelos Soulintzis, Software Engineer

Introduction to Pydantic

Pydantic is a Python library created by Samuel Colvin that simplifies the process of data validation. It is a versatile tool that can be utilized in various contexts, such as building APIs, working with databases, and handling data within projects. The library features a simple and intuitive syntax, allowing for easy definition and validation of data models. This includes the ability to specify types, default values, and constraints directly within the code, making it both understandable and maintainable.

Despite its simplicity, Pydantic is also highly flexible and powerful, providing built-in validators and constraints as well as the capability to create custom validators for further data definition and validation. The library is accessible to both experienced Python developers and those new to the language, offering something for everyone. Join us as we explore the world of Pydantic.

Pydantic Features

Pydantic is a Python library that provides a range of data validation and parsing features. Some of the main features of Pydantic include:

1. Data validation: Pydantic includes a validation function that automatically checks the types and values of class attributes, ensuring that they are correct and conform to any specified constraints. Custom validation rules using Python-type hinting are also supported, allowing users to define additional constraints on their data.

2. Support for complex data types: Supports a wide range of data types, including lists, dictionaries, and nested models, making it easy to define and validate complex data structures.

3. Efficient and lightweight: Pydantic is designed to be fast and efficient, with a small codebase and minimal dependencies. It is also well-documented and easy to use, making it a popular choice for data validation and parsing in Python.

4. Open source: Pydantic is an open-source library, licensed under the MIT license, and is actively developed and maintained by a community of contributors.

Pydantic vs Python Data Classes

Python data classes are a way to define classes primarily used to store data. They were introduced in Python 3.7 by the @dataclass decorator. When applied to a class, this decorator automatically generates a default __init__ method, a default __repr__ method, and default comparison methods, which simplifies the definition of classes that only need to store data. Additionally, the dataclasses module provides helper functions like dataclasses.asdict() and dataclasses.replace() that can be used to convert a class instance to a dictionary or to create a new instance from the class with updates to the fields respectively. This makes data classes a convenient way to create simple classes that only need to store data and don't require any other behavior.

Pydantic is a library that builds on top of Python data classes and adds additional functionality for data validation and parsing. Pydantic provides a special class BaseModel that can be used to define data models and their validation rules. The validation rules can be defined by setting types and constraints on the class attributes and can also be customized with additional rules using Python type hinting. Also provides support for parsing data from sources such as JSON, and can convert the parsed data into an instance of the defined data model. Additionally, can generate a JSON schema from the data model, which can be used to validate input data based on the schema.

Overall, Pydantic is a library that adds additional functionality for data validation and parsing to Python. While Python data classes provide a convenient syntax for defining simple classes that store data, Pydantic offers a more powerful and flexible solution for validating and parsing data in Python applications. BaseModel class can be used to define data models and their validation rules, it can validate input data and parse it into a Python object and also serialize it to JSON and other formats. This makes Pydantic a valuable tool for ensuring data integrity and consistency in any Python application that deals with data.

Examples of using the Pydantic Library

One of the main benefits of using Pydantic is that it allows you to define the structure of your data in a declarative way, using Python’s type hints. This means that you can specify the types, default values, and constraints of your data directly in the code, which makes it easy to understand and maintain.

For example, let’s say you want to define a simple data model for a User, with fields for their username, age, email and password . With Pydantic, you can define this model like this:

from pydantic import BaseModelclass User(BaseModel):
 username: str
 age: int
 email: str
 password: str

Each field is annotated with a type hint, which specifies the type of data that it can hold. In this case, username , email and password are strings, while age is an integer.

Pydantic also provides a number of built-in validators and constraints that you can use to further define and validate your data. For example, you can specify that a field is required, or that it must match a certain pattern. You can also specify default values for fields, which can be helpful when working with optional data.

Here’s an example of how you might use some of these features to further define the User model:

class User(BaseModel):
 username: str
 age: conint(gt=18)
 email: EmailStr
 password: constr(min_length=8, max_length=16)
 phone: Optional[str] = 'Unknown'

In this example, we’ve used the constr validator to define constrains for the age and password field. that it must be at least 8 characters long. The age field is defined as a conint type with a condition that its value must be greater than 18, and the password field is defined as a constr type with two conditions, the value must be at least 8 characters long and at most 16 characters long. We've also used the EmailStr type to specify that the email field must contain a valid email address. Finally, a phonefield has been added which is Optional and has a default value of ‘Unknown’.

In addition to the built-in validators and constraints provided, you can also create your own custom validators. This can be helpful when you need to enforce more specific or complex rules on your data that are not covered by built-in validators.

To create a custom validator, you can define a function that takes in a value and performs any necessary validation checks. If the value is valid, the function should return it. If the value is not valid, the function should raise a ValidationError with a message explaining the problem.

Here’s an example of using custom validators for the User model, similar to the previous example, and showing how data are passed into the Pydantic class for validation and what is the Pydantic output.

import re
from pydantic import BaseModel, validatorclass User(BaseModel):
 username: str
 age: int
 email: str
 password: str
 @validator("age")
 @classmethod
 def validate_age(cls, value):
 if value < 18:
 raise ValueError("User must be adult")
 return value
 @validator("email")
 @classmethod
 def validate_email(cls, value):
 if not bool(re.fullmatch(r'[\w.-]+@[\w-]+\.[\w.]+', value)):
 raise ValueError("Email is invalid")
 return value 
 @validator("password")
 @classmethod
 def validate_password(cls, value):
 password_length = len(value)
 if password_length < 8 or password_length > 16:
 raise ValueError("The password must be between 8 and 16 characters long")
 return value
# Valid User
valid_user = {
 'username': 'test_name',
 'age': 20,
 'email': 'name@test.gr',
 'password': '123456789'
}
# Invalid User
invalid_user = {
 'username': 'test_name',
 'age': 16,
 'email': 'name_test.gr',
 'password': '1234'
}
try:
 valid_result = User(**valid_user)
 print(valid_result)
 # Valid Output: username='test_name' age=20 email='name@test.gr' password='123456789'
 invalid_result = User(**invalid_user)
except ValueError as e:
 print(e.errors())
 # Invalid Output: 
 [
 {
 "loc":"(""age",")",
 "msg":"User must be adult",
 "type":"value_error"
 },
 {
 "loc":"(""email",")",
 "msg":"Email is invalid",
 "type":"value_error"
 },
 {
 "loc":"(""password",")",
 "msg":"The password must be between 8 and 16 characters long",
 "type":"value_error"
 }
 ]

Custom validators can be very powerful and can help you create data models that are tailored to your specific needs. Whether you’re working with simple data types or more complex structures, custom validators can help you define and validate your data in a flexible and reliable way.

Pydantic and FastAPI

Pydantic is a popular choice for defining data models in FastAPI, a high-performance Python web framework for building APIs. FastAPI is built on top of Starlette, a lightweight ASGI framework, and integrates seamlessly with Pydantic to provide a fast and easy-to-use way to build APIs.

One of the main benefits of using Pydantic with FastAPI is that it allows you to define your API’s request and response data in a simple and intuitive way. You can use Pydantic’s data models to define the structure of your request and response data, and FastAPI will automatically validate and convert the data to and from JSON as needed.

For example, let’s say you want to build an API that allows users to create, retrieve, update, and delete tasks. With FastAPI and Pydantic, you can define the data models for the tasks like this:

Let’s use the User model from the previous section to define the FastAPI routes for creating, updating, retrieving and deleting a User.

from fastapi import FastAPIapp = FastAPI()
@app.post("/users")
def create_user(user: User):
 pass
@app.get("/users/{username}")
def get_user(username: str):
 pass
@app.put("/users/{username}")
def update_user(username: str, user: User):
 pass
@app.delete("/users/{username}")
def delete_user(username: str):
 pass

In this example, we’ve defined four API routes: one for creating a new user, one for retrieving an existing user, one for updating a user, and one for deleting a user. Each route takes in a request payload (if applicable) and uses Pydantic’s data models to validate and convert the data as needed.

With FastAPI and Pydantic, you can quickly and easily build APIs that are simple, intuitive, and easy to work with. Whether you’re building a small API or a large-scale application, FastAPI and Pydantic can help you define and validate your data in a reliable and efficient way.

Comparison to other data validation libraries

There are several other Python libraries that provide similar functionality to Pydantic, such as Marshmallow, Cerberus, and Voluptuous. Here are some ways in which Pydantic compares to these libraries:

Marshmallow is a library for data serialization and validation, it allows to define fields, validations, and other complex data structures which can be used to validate, deserialize and serialize data in various formats.

Cerberus is another library for data validation in Python. It has a declarative syntax similar to marshmallow, but it is focused more on data validation than serialization. One advantage of Cerberus is that it is lightweight and has a smaller footprint than Pydantic or marshmallow. However, Pydantic provides a wider range of features and is more powerful overall.

Voluptuous is a Python library for data validation that is similar to Cerberus in terms of its focus on data validation. It has a declarative syntax and is lightweight, but it does not provide as many features as Pydantic or marshmallow.

Overall, Pydantic is a more feature-rich library compared to Cerberus and voluptuous, and it provides a different approach to data validation and serialization compared to marshmallow. It may be a good choice depending on the specific needs of your application.

Conclusion

In Project Agora, we use Pydantic as our preferred data validation library for REST APIs because of its ease of use, flexibility, and power. With Pydantic’s simple and intuitive syntax, it is easy to define and validate data models, ensuring that the data received by the API is consistent and accurate. Additionally, Pydantic’s built-in validators and constraints, as well as the ability to create custom validators, provide a robust means of defining and validating data. This leads to more reliable and secure APIs that can handle and process data with confidence.

The official Pydantic documentation can be found here.

Keep abreast of our latest developments and insights by subscribing to our Project Agora blog. If you’re inspired to join our team, be sure to check out our current job openings on our career page.

Articles: Pydantic in Python, AI applications/examples