An efficient, flexible and fast toolkit for managing your data

Unleash the full power of your hardware to snappily process your extremely large multidimensional datasets in memory or on disk.

Why ironArray?

We do believe that Big Data handling can play better with our environment by making a more intelligent use of the energy, so we are making ironArray in such a way that it allows computations with large datasets to make a more effective use of modern, cost-effective multi-core CPUs and fast, local storage.

Highlights​

  • High performance matrix and vector computations
  • Automatic data compression in-memory or on disk
  • Dynamically tuned performance optimizations using artificial intelligence that leverage your specific CPUs, caches, memory and disks
  • User defined functions

Open-Source pedigree

The data container is based on the Blosc2 library and format and leverages the Caterva library for fast manipulation of multidimensional data. Get snappy slices of your datasets, no matter how big they are.

Open data format

ironArray uses the simple, expandable, well documented and open-source format from the Blosc2 library. Use contiguous or sparse storage, both in-memory or on-disk, with no compromises in speed.

How it works

ironArray organizes your data into chunks that fit into the last-level cache of your CPU and blocks that fit in first-levels cache. Then uses standard map, reduce, filter, and collect algorithms to perform calculations on large arrays directly in the high-speed CPU cache.

As a result, computations can make a much better use of scarce memory bandwidth of modern computers.  Furthermore, if performance computations are limited by memory bandwidth, this limit can even be surmounted!

Just express your performance and storage needs

And let ironArray do the rest.  Based on your preferences, ironArray will tune your configuration using state-of-the-art artificial intelligence algorithms, then create new arrays and perform calculations using the applied optimizations.

For example, you can set a preference for high compression ratios, or for high-speed computation, or for some balance between the two.  Based on that, ironArray will adapt partition sizes, the number of threads and other parameters to better adapt to your requirements.

What ironArray can do for you

Reduce memory consumption

Minimize energy consumption

Reduce cluster size​

Minimize infrastructure cost

Reduce complexity​

Minimize development and maintenance efforts

Improve slicing performance

Be more selective on fetching the data that you want

Leverage compression automatically

Do more with less

Compute faster

Unleash the full capabilities of your hardware