Expression Evaluation (In-Memory)¶
ironArray has a strong support for expression evaluation. Things like sums, products, divisions and a pretty complete range of transcendental functions (e.g.
tanh…) have been implemented so as to guarantee an efficient evaluation in (large) arrays. Performance comes from a balance between:
Use of an internal just in time compiler.
Use of multi-threading capabilities.
Leveraging the 2-level partitioning in ironArray arrays so that most of the computation intensive happens inside private caches (L1, L2), which benefits multi-threading performance.
You can access to the powerful evaluation capabilities in ironArray in different ways, which we are going to show in this tutorial. In order to do that, we are going to make use of the dataset that we created during our reductions tutorial. Let’s go:
%load_ext memprofiler %matplotlib inline import matplotlib.pyplot as plt import iarray as ia
Now, in order to evaluate some expressions on this, let’s put the data for each month on a different array:
%%mprof_run -i 0.1 load precip1 = ia.load("precip1.iarr") precip2 = ia.load("precip2.iarr") precip3 = ia.load("precip3.iarr")
memprofiler: used 842.05 MiB RAM (peak of 842.05 MiB) in 0.2273 s, total RAM usage 1066.90 MiB
Here we can see how the memory is being used during the load process:
%mprof_plot load -t "Memory usage for loading ironArray data"