What is FFI (Foreign Function Interface) — An Intuitive Explanation
To put it simply, Foreign Function Interface (FFI) is a mechanism for calling functions from one programming language (say C) in another programming language (say Python). This is incredibly useful as it allows you to evade the often burdensome but common language trade off — programming time vs efficiency.
How does it work?
FFI isn’t magic, but to understand how it works, we must first understand how programs are compiled.
Let’s take an example program written in C — add.c
int add(int a, int b) {
return a + b;
}
int main() {
int x = add(1,3);
return 0;
}When you compile this program using a C compiler, say GCC for instance, this c code becomes what we call “machine code”. This is a set of binary instructions which your CPU knows how to execute step by step. The “add” function is just some set of these instructions in a certain memory location that are branched to when needed and executed. By branching, I mean the CPU literally “jumps” to a new memory location and executes the instructions there. This is all functions are.
FFI simply allows one program to utilize this set of instructions, even if they weren’t necessarily generated by the original language of the program.
So in the case of our program, we would be able to use this “add” function written in C in another language, as long as this language knows where to “branch” to. Of course, we can’t do this out of the box. After all, if we compile this program normally, the OS will assume it is a singular process, and so the binary instructions associated with the “add” function will only be accessible within the program containing the function definition.
DLLs and Shared Libraries
DLLs (Dynamically Linked Libraries, Windows) and Shared Libraries (Unix) are OS-level entities that are similar to regular programs in that they are compiled machine code. However, they differ in that we can use their machine code in other programs. In other words, the OS “exposes” their binary instructions. They can be created usually by specifying a flag when compiling our program, such as -shared in the case of GCC, which is specifically for creating a shared library.
Then, we can use these shared libraries in programs written in other languages, including Python. For instance, let’s take the add example above. Say we compile the above program into a shared library called add.so. Using a Python module called ctypes, we can do:
import ctypes
# Load the shared library
add_lib = ctypes.CDLL(‘./add.so’)
# Call the function
result = add_lib.add(2, 3)
print(result)When this Python program calls add_lib.add, ctypes knows to jump to and execute the correct binary instructions for the “add” function. It also takes care of ensuring correct types — both for parameters and return values — are sent to and from the function. Other languages may implement similar modules or libraries to ctypes. The commonality is that such a module or library must be able to look up the address of functions, ensure correct data flow, and also return the results properly.
What about static libraries?
You can also use FFI with static libraries, but these have to be “built into” your program. For instance, in the Python example, one would have to combine the Python compiler itself (which compiles source code to bytecode) with the shared library. In compiled languages (unlike Python, which is not usually involved in the linking process), this is a little easier as you can simply link the two separate programs together. However, using static libraries would require you to recompile your entire application if the library changes.
That’s it for FFI! Of course there is a lot more technical jumbo to all this, so I encourage you to research more. Hopefully this article prepares you for such research, and at least leaves you with an intuitive understanding of why and how things work.
