Are you a Python developer with a C or C++ library you’d like to use from Python? If so, then Python bindings allow you to call functions and pass data from Python to C or C++, letting you take advantage of the strengths of both languages. Throughout this tutorial, you’ll see an overview of some of the tools you can use to create Python bindings.
In this tutorial, you’ll learn about:
- Why you want to call C or C++ from Python
- How to pass data between C and Python
- What tools and methods can help you create Python bindings
This tutorial is aimed at intermediate Python developers. It assumes basic knowledge of Python and some understanding of functions and data types in C or C++. You can get all of the example code you’ll see in this tutorial by clicking on the link below:
Get Sample Code: Click here to get the sample code you’ll use to learn about Python Bindings in this tutorial.
Let’s dive into looking at Python bindings!
Take the Quiz: Test your knowledge with our interactive “Python Bindings: Calling C or C++ From Python” quiz. You’ll receive a score upon completion to help you track your learning progress:
Interactive Quiz
Python Bindings: Calling C or C++ From PythonTest your understanding of Python bindings, from marshaling data and managing memory to calling C and C++ with ctypes, CFFI, PyBind11, and Cython.
Python Bindings Overview
Before you dive into how to call C from Python, it’s good to spend some time on why. There are several situations where creating Python bindings to call a C library is a great idea:
-
You already have a large, tested, stable library written in C++ that you’d like to take advantage of in Python. This may be a communication library or a library to talk to a specific piece of hardware. What it does is unimportant.
-
You want to speed up a particular section of your Python code by converting a critical section to C. Not only does C have faster execution speed, but it also allows you to break free from the limitations of the GIL, provided you’re careful.
-
You want to use Python test tools to do large-scale testing of their systems.
All of the above are great reasons to learn to create Python bindings to interface with your C library.
Note: Throughout this tutorial, you’ll be creating Python bindings to both C and C++. Most of the general concepts apply to both languages, and so C will be used unless there’s a specific difference between the two languages. In general, each of the tools will support either C or C++, but not both.
Let’s get started!
Marshalling Data Types
Wait! Before you start writing Python bindings, take a look at how Python and C store data and what types of issues this will cause. First, let’s define marshaling. This concept is defined by Wikipedia as follows:
The process of transforming the memory representation of an object to a data format suitable for storage or transmission. (Source)
For your purposes, marshaling is what the Python bindings are doing when they prepare data to move it from Python to C or vice versa. Python bindings need to do marshaling because Python and C store data in different ways. C stores data in the most compact form in memory possible. If you use an uint8_t, then it will only use 8 bits of memory total.
In Python, on the other hand, everything is an object. This means that each integer uses several bytes in memory. How many will depend on which version of Python you’re running, your operating system, and other factors. This means that your Python bindings will need to convert a C integer to a Python integer for each integer passed across the boundary.
Other data types have similar relationships between the two languages. Let’s look at each in turn:
-
Integers store counting numbers. Python stores integers with arbitrary precision, meaning that you can store very, very, large numbers. C specifies the exact sizes of integers. You need to be aware of data sizes when you’re moving between languages to prevent Python integer values from overflowing C integer variables.
-
Floating-point numbers are numbers with a decimal place. Python can store much larger (and much smaller) floating-point numbers than C. This means that you’ll also have to pay attention to those values to ensure they stay in range.
-
Complex numbers are numbers with an imaginary part. While Python has built-in complex numbers, and C has complex numbers, there’s no built-in method for marshaling between them. To marshal complex numbers, you’ll need to build a
structorclassin the C code to manage them. -
Strings are sequences of characters. For being such a common data type, strings will prove to be rather tricky when you’re creating Python bindings. As with other data types, Python and C store strings in quite different formats. (Unlike the other data types, this is an area where C and C++ differ as well, which adds to the fun!) Each of the solutions you’ll examine have slightly different methods for dealing with strings.
-
Boolean variables can have only two values. Since they’re supported in C, marshaling them will prove to be fairly straightforward.
Besides data type conversions, there are other issues you’ll need to think about as you build your Python bindings. Let’s keep exploring them.
Understanding Mutable and Immutable Values
In addition to all of these data types, you’ll also have to be aware of how Python objects can be mutable or immutable. C has a similar concept with function parameters when talking about pass-by-value or pass-by-reference. In C, all parameters are pass-by-value. If you want to allow a function to change a variable in the caller, then you need to pass a pointer to that variable.
You might be wondering if you can get around the immutable restriction by simply passing an immutable object to C using a pointer. Unless you go to ugly and non-portable extremes, Python won’t give you a pointer to an object, so this just doesn’t work. If you want to modify a Python object in C, then you’ll need to take extra steps to achieve this. These steps will be dependent on which tools you use, as you’ll see below.
So, you can add immutability to your checklist of items to consider as you create Python bindings. Your final stop on the grand tour of creating this checklist is how to handle the different ways in which Python and C deal with memory management.
Read the full article at https://realpython.com/python-bindings-overview/ »
[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]
