Lazy Expressions in Caterva2
What is Caterva2?
Caterva2 is a Free/Open Source distributed system written in Python meant for sharing Blosc2 datasets (either native or converted on-the-fly from HDF5) among different hosts. It uses a publish–subscribe messaging pattern where the data of a publisher can be replicated by an unlimited amount of subscribers. Also, every subscriber exposes a REST interface that allows clients to access the datasets.
Let's suppose that we have a large dataset that we want to share with a group of people. Instead of sending the data to each person individually, we can use Caterva2 to publish the data once and allow multiple subscribers to access it. This way, we can save time and resources by avoiding the need to send the data to each person separately.
Lazy expressions in Caterva2
Besides the data sharing utility, Caterva2 can also perform operations on the data via Python-Blosc2 v3. These operations range from arithmetic expressions to reductions, filters and broadcasting, and are performed lazily, that is, only when a part of the result is needed.