Julia vs Python

Should we ditch Python and other languages in favor of Julia for technical computing?  That’s certainly a thought that comes to mind when one looks at the benchmarks on http://julialang.org/.  Python and other high level languages are way behind in term of speed.  The first question that came to my mind was different however: did the Julia team wrote Python benchmarks the best way for Python?

My take on this kind of cross language comparison is that the benchmarks should be defined by tasks to perform, then have language experts write the best code they can to perform these tasks.  If the code is all written by one language team, then there is a risk that other languages aren’t used at best.

One thing the Julia team did right is to publish on github the code they used.  In particular, the Python code can be found here.

A first look at this code confirms the bias I was afraid of.  The code is written in a C style with heavy use of loops over arrays and lists.  This is not the best way to use Python.

I won’t blame the Julia team, as I have been guilty of the exact same bias.  But I learned the hard lesson: loops on arrays or lists should be avoided at almost any cost as they are really slow in Python, see Python is not C.

Given this bias towards C style, the interesting question (to me at least) is whether we can improve these benchmarks with a better use of Python and its tools?

Before I give the answer below, let me say that I am in no way trying to downplay Julia.  It is certainly a language worth monitoring as it is further developed and improved.  I just want to have a look at the Python side of things.  Actually, I am using this as an excuse to explore various Python tools that can be used to make code run faster.

In what follows I use Python 3.5.1 with Anaconda on a Windows machine.  The notebook containing the complete code for all benchmarks below is available on github and on nbviewer.

Comments on various social media make me add this: I am not writing any C code here: if you’re not convinced, then try to find any semicolon. All the tools used in this blog run in the standard CPython implementation available in Anaconda or other distributions.  All the code below runs in a single notebook.  I tried to use the Julia micro performance file from github but it does not run as is with Julia 0.4.2.  I had to edit it and replace @timeit by @time to make it run.  I also had to add calls to the timed functions before timing them, otherwise the compilation time was included.  I ran it with the Julia command line interface, on the same machine as the one used to run Python.

Source: How To Make Python Run As Fast As Julia (IT Best Kept Secret Is Optimization)