Powerful Tool for Big Data Problems and Logical Inference

BitMagic Library helps to develop high-throughput intelligent search systems, promote combination of hardware optimizations and on the fly compression to fit inverted indexes and binary fingerprints into memory, minimize disk and network footprint.

Functions

• compressed bit-vector container, implements random access methods, with range of set-algebraic functions, ranks, find and traverse methods, STL-style iterators
• set algebraic operations: AND, OR, XOR, MINUS for bit-vectors and integer sets. Interoperable with low level C arrays and STL compatible containers (via iterators).
• serialization/hybernation of containers into compressed BLOBs for database persistence or in-memory compression
• memory management with focus on optimization (avoiding) allocations/de-allocations, minimization of heap fragmentation, custom allocators.
• set algebraic operations on compressed bit-vector BLOBs
• statistical engine to efficiently construct binary similarity and distance metrics (Tevrsky, Hamming, Tanimoto, Dice or your own)
• containers for sparse vectors and collections for native integer types. Works throug bit-transposition and compression of each separate bit-plain. Supports for NULL semantics. Can be used for memory-compresses vector/columnar search systems with focus on memory efficiency
• algorithms on sparse vectors: dynamic range clipping (work in progress!)
• functional operations on integer sets (theory of groups): translations between sets, mathematical images (work in progress!).
• binary compressed matrices for ER-operations, materialized joins, one-to-many and many-to-many relationships, materialized RDBMS joins, graphs, etc. (work in progress!)
• portable C-library layer as a bridge to Python, Java, .Net (work in progress!)