Error handling

ironArray has a sophisticated error handling machinery allowing to catch errors at multiple levels. To see how all this works, let’s try to build a block iterator with a different shape than the chunk shape in an array.

import iarray as ia

a = ia.empty(shape=[10, 10, 10], urlpath="array.iarr", mode="w")
itershape = [4, 5, 6]
a.iter_write_block(itershape)

---------------------------------------------------------------------------
IArrayError                               Traceback (most recent call last)
/tmp/ipykernel_80559/849519659.py in <module>
      1 a = ia.empty(shape=[10, 10, 10], urlpath="array.iarr", mode="w")
      2 itershape = [4, 5, 6]
----> 3 a.iter_write_block(itershape)

~/iarray/iron-array-python/iarray/iarray_container.py in iter_write_block(self, iterblock)
    134             else:
    135                 iterblock, _ = ia.partition_advice(self.shape)
--> 136         return ext.WriteBlockIter(self, iterblock)
    137
    138     def __getitem__(self, key):

iarray_ext.pyx in iarray.iarray_ext.WriteBlockIter.__cinit__()

iarray_ext.pyx in iarray.iarray_ext.iarray_check()

IArrayError: b'NOT VALID - 0x800a000000938414 - error=1,ver=0,rev=10,os=0,neg=1,adj=147,subject=1044,code=9633792,ubits=0x0'

The ironArray error handling machinery raised a Python error called IArrayError. This error returns a general error message. In this particular case, the error is NOT VALID but it does not indicate much more beyond this.

In order to obtain more detailed information, we can set the environment variable INAC_TRACE="iarray.error". With this, ironArray will show us a more detailed error; in particular, it will provide us with a trace of what’s going on in the underlying C library.

To better leverage this functionality inside a Jupyter notebook, it is convenient to use the wurlitzer extension to redirect stdout and stderr handlers to the notebook:

%load_ext wurlitzer

import os
os.environ['INAC_TRACE'] = 'iarray.error'

and, after doing this, we can get more info on the ironArray core library (written in C):

a.iter_write_block(itershape)

---------------------------------------------------------------------------
IArrayError                               Traceback (most recent call last)
/tmp/ipykernel_80559/4188138722.py in <module>
----> 1 a.iter_write_block(itershape)

~/iarray/iron-array-python/iarray/iarray_container.py in iter_write_block(self, iterblock)
    134             else:
    135                 iterblock, _ = ia.partition_advice(self.shape)
--> 136         return ext.WriteBlockIter(self, iterblock)
    137
    138     def __getitem__(self, key):

iarray_ext.pyx in iarray.iarray_ext.WriteBlockIter.__cinit__()

iarray_ext.pyx in iarray.iarray_ext.iarray_check()

IArrayError: b'NOT VALID - 0x800a000000938414 - error=1,ver=0,rev=10,os=0,neg=1,adj=147,subject=1044,code=9633792,ubits=0x0'

[iarray.error] - /home/faltet2/iarray/iron-array-python/iarray/iarray-c-develop/src/iarray_iterator.c:497
The iterator iter_blockshape must be equal to the container chunkshape

Now, we get the error message The iterator iter_blockshape must be equal to the container chunkshape that already gives us a better idea of what we were doing wrong.

In addition, we can see in which lines of the ironArray core code the error has occurred. In this case, the trace is:

• iarray/iarray-c-develop/src/iarray_iterator.c:497

While those will be mostly useful for developers, it is always interesting that you can attach these when you file possible bug reports.

That’s all. Whenever you would like more info on your errors, remember to activate the INAC_TRACE environment variable.

previous

Expression Evaluation (On-Disk)

next

Matrix multiplication