How Slow is Python Compared to C
To compare the computation performance between Python and C languages, let’s do a loop for sum in one second. The code itself is pretty much self-explanatory.
time python python_loop.py 10000000
It is 10 millions loops in a second for Python. It sounds not bad.
Compile in normal mode:
gcc c_loop.c -o c_loop
time ./c_loop 450000000
It is 450 million loops in a second, which is 45 times faster than Python.
Furthermore, C can be compiled in optimized mode for a better performance.
gcc -O3 c_loop.c -o c_loop
time ./c_loop 450000000
Yes, it is unbelievable! It is 1000 times faster than normal mode, and 45,000 times faster than Python.
Note that the optimization is not always 1000 times faster for all C code, but it should be at least 10 times faster, so it is still hundreds of times faster than Python. Also bear in mind that the speed difference between C and Python varies on different code.
We use node.js to run the code with 1000,000,000 loops.
time node loop.js
That is 100 times faster than Python.
AWS Lightsail VM: 1CPU, 1G RAM
Ubuntu 16.04 64bit
gcc version 7.4.0
Give Up Python?
Should I give up Python then? No.
Despite its slowness Python is getting more, not less, popular because of its simplicity, IMHO.
“At Instagram, our bottlenect is development velocity, not pure code execution.” From PyCon2017
For most IT companies human resources are the biggest cost, not the machines. If Python is slower, just buy more virtual machines in the cloud. With the median IT hourly wage $34 in USA, if you save 1 hour by using Python, you can hire 7 VMs (1CPU, 1G Memory) for a month!
Moreover, in most cases you don’t see the performance impact. If it is all done within 1 second, it makes no difference from an end user perspective.
If performance is really a concern as your business grows, following are some of the options.
- Build the heavy function as a C shared library and wrap with Python using ctypes. Numpy is a use case of this approach. Without this, machine learning using Python is impossible.
- Use Cython. Cython simplifies writing C extension for Python. It is like a bridge between C and Python, and you write C extension in a Python-like fashion.
- Use Pypy. Pypy gives you a boost of 4.2 times speed without changing a line of your python code. Just run with pypy instead of python command.
- Build the heavy function in C as a socket-based microservice, a separate executable, and call it from Python. Typical interfaces between C and Python programs are REST API, ZeroMQ, Google gRPC etc. The advantage of this approach is that you can deploy C and Python programs either in the same machine or distributed systems like containers.
A Taste of Cython
Cython actually converts the Cython file to C source file, then builds a shared library.
First, we write a Cython file(.pyx) for the loop function. Don’t worry about Cython syntax. If you know Python, you already knows the basic Cython. They are the same except that you define the static type for variables in this case. We use integer
int and keyword
Second, we create a normal Python file setup_cython.py to call Cython
cythonize function to convert cython .pyx to C files and build a shared library (.so).
Run the cython setup as follows:
python setup_cython.py build_ext --inplace
And you will see two new files generated:
This is the converted C file from the Cython .pyx file. If you open and look at it carefully, you can see the line to line mapping from Cython to C.
This is the created shared library file which can be called from Python. But Cython makes it easier to use it as follows.
Finally, we can write a main Python file and import the cython_loop module just like a normal Python file.
Note: It actually calls cython_loop.cpython-37m-x86_64-linux-gnu.so.
Let’s run it and see what performance we get.
time python cython_call.py 450000000
sum: 450000000real 0m0.025s
It is 450 millions of loops in 0.025s, which is not far from C and much faster (1,800 times) than pure Python.
Isn’t it your solution? Still Python but in speed!
A Taste of Pypy
Let’s run the same Python for 10 millions loop, which is 1 second with CPython interpreter.
Simply install pypy and run with pypy command.
time pypy loop.py 10000000real 0m0.058s
Note: The 1st run may be slower, but is still faster than CPython.
 PyCon 2020, Anthony Shaw - Why is Python slow