How Slow is Python Compared to C

45,000 times slower!

Photo by Ben White on Unsplash

To compare the computation performance between Python and C languages, let’s do a loop for sum in one second. The code itself is pretty much self-explanatory.

Python

Python code

Run:

time python python_loop.py 10000000
real    0m1.044s
user    0m1.004s
sys     0m0.004s

It is 10 millions loops in a second for Python. It sounds not bad.

C

Code Credits: One second code

Compile in normal mode:

gcc c_loop.c -o c_loop

Run:

time ./c_loop 450000000
s: 450000000
real    0m1.032s
user    0m1.008s
sys     0m0.008s

It is 450 million loops in a second, which is 45 times faster than Python.

Furthermore, C can be compiled in optimized mode for a better performance.

Compile:

gcc -O3 c_loop.c -o c_loop

Run:

time ./c_loop 450000000
real    0m0.001s
user    0m0.000s
sys     0m0.000s

Yes, it is unbelievable! It is 1000 times faster than normal mode, and 45,000 times faster than Python.

Note that the optimization is not always 1000 times faster for all C code, but it should be at least 10 times faster, so it is still hundreds of times faster than Python. Also bear in mind that the speed difference between C and Python varies on different code.

Javascript

Okay, let’s make a fair comparison between script (interpreted) languages Python v.s. Javascript.

Javascript code

We use node.js to run the code with 1000,000,000 loops.

time node loop.js

Result:

real    0m1.048s
user    0m1.036s
sys     0m0.008s

That is 100 times faster than Python.

Test Environment

AWS Lightsail VM: 1CPU, 1G RAM
Ubuntu 16.04 64bit
Python 3.7
gcc version 7.4.0
node.js 14.1.0

Photo by Tonik on Unsplash

Give Up Python?

Should I give up Python then? No.

Despite its slowness Python is getting more, not less, popular because of its simplicity, IMHO.

“At Instagram, our bottlenect is development velocity, not pure code execution.” From PyCon2017

For most IT companies human resources are the biggest cost, not the machines. If Python is slower, just buy more virtual machines in the cloud. With the median IT hourly wage $34 in USA, if you save 1 hour by using Python, you can hire 7 VMs (1CPU, 1G Memory) for a month!

Moreover, in most cases you don’t see the performance impact. If it is all done within 1 second, it makes no difference from an end user perspective.

Solution

If performance is really a concern as your business grows, following are some of the options.

• Build the heavy function as a C shared library and wrap with Python using ctypes. Numpy is a use case of this approach. Without this, machine learning using Python is impossible.
• Use Cython. Cython simplifies writing C extension for Python. It is like a bridge between C and Python, and you write C extension in a Python-like fashion.
• Use Pypy. Pypy gives you a boost of 4.2 times speed without changing a line of your python code. Just run with pypy instead of python command.
• Build the heavy function in C as a socket-based microservice, a separate executable, and call it from Python. Typical interfaces between C and Python programs are REST API, ZeroMQ, Google gRPC etc. The advantage of this approach is that you can deploy C and Python programs either in the same machine or distributed systems like containers.

A Taste of Cython

Cython actually converts the Cython file to C source file, then builds a shared library.

First, we write a Cython file(.pyx) for the loop function. Don’t worry about Cython syntax. If you know Python, you already knows the basic Cython. They are the same except that you define the static type for variables in this case. We use integer int and keyword cdef.

Cython file for the heavy function

Second, we create a normal Python file setup_cython.py to call Cython cythonize function to convert cython .pyx to C files and build a shared library (.so).

setup file for Cython

Run the cython setup as follows:

python setup_cython.py build_ext --inplace

And you will see two new files generated:

• cython_loop.c
  This is the converted C file from the Cython .pyx file. If you open and look at it carefully, you can see the line to line mapping from Cython to C.
• cython_loop.cpython-37m-x86_64-linux-gnu.so
  This is the created shared library file which can be called from Python. But Cython makes it easier to use it as follows.

Finally, we can write a main Python file and import the cython_loop module just like a normal Python file.
Note: It actually calls cython_loop.cpython-37m-x86_64-linux-gnu.so.

The main Python file

Let’s run it and see what performance we get.

time python cython_call.py 450000000
sum: 450000000
real    0m0.025s
user    0m0.020s
sys     0m0.000s

It is 450 millions of loops in 0.025s, which is not far from C and much faster (1,800 times) than pure Python.

Isn’t it your solution? Still Python but in speed!

A Taste of Pypy

Let’s run the same Python for 10 millions loop, which is 1 second with CPython interpreter.

Python code

Simply install pypy and run with pypy command.

time pypy loop.py 10000000
real    0m0.058s
user    0m0.040s
sys     0m0.016s

Note: The 1st run may be slower, but is still faster than CPython.

Useful References

[1] PyCon 2020, Anthony Shaw - Why is Python slow