Safe Haskell | None |
---|---|
Language | Haskell2010 |
Frames.InCore
Description
Efficient in-memory (in-core) storage of tabular data.
- type family VectorFor t :: * -> *
- type VectorMFor a = Mutable (VectorFor a)
- initialCapacity :: Int
- type family VectorMs m rs where ...
- type family Vectors rs where ...
- class RecVec rs where
- inCoreSoA :: forall m rs. (PrimMonad m, RecVec rs) => Producer (Record rs) m () -> m (Int, Rec ((->) Int) rs)
- inCoreAoS :: (PrimMonad m, RecVec rs) => Producer (Record rs) m () -> m (FrameRec rs)
- inCoreAoS' :: (PrimMonad m, RecVec rs) => (Rec ((->) Int) rs -> Rec ((->) Int) ss) -> Producer (Record rs) m () -> m (FrameRec ss)
- toAoS :: Int -> Rec ((->) Int) rs -> FrameRec rs
- inCore :: forall m n rs. (PrimMonad m, RecVec rs, Monad n) => Producer (Record rs) m () -> m (Producer (Record rs) n ())
- toFrame :: (Foldable f, RecVec rs) => f (Record rs) -> Frame (Record rs)
- filterFrame :: RecVec rs => (Record rs -> Bool) -> FrameRec rs -> FrameRec rs
Documentation
type VectorMFor a = Mutable (VectorFor a) #
The mutable version of VectorFor
a particular type.
initialCapacity :: Int #
Since we stream into the in-memory representation, we use an exponential growth strategy to resize arrays as more data is read in. This is the initial capacity of each column.
Tooling to allocate, grow, write to, freeze, and index into records of vectors.
Methods
allocRec :: PrimMonad m => proxy rs -> m (Record (VectorMs m rs)) #
freezeRec :: PrimMonad m => proxy rs -> Int -> Record (VectorMs m rs) -> m (Record (Vectors rs)) #
growRec :: PrimMonad m => proxy rs -> Record (VectorMs m rs) -> m (Record (VectorMs m rs)) #
writeRec :: PrimMonad m => proxy rs -> Int -> Record (VectorMs m rs) -> Record rs -> m () #
indexRec :: proxy rs -> Int -> Record (Vectors rs) -> Record rs #
produceRec :: proxy rs -> Record (Vectors rs) -> Rec ((->) Int) rs #
inCoreSoA :: forall m rs. (PrimMonad m, RecVec rs) => Producer (Record rs) m () -> m (Int, Rec ((->) Int) rs) #
Stream a finite sequence of rows into an efficient in-memory
representation for further manipulation. Each column of the input
table will be stored optimally based on its type, making use of the
resulting generators a matter of indexing into a densely packed
representation. Returns the number of rows and a record of column
indexing functions. See toAoS
to convert the result to a Frame
which provides an easier-to-use function that indexes into the
table in a row-major fashion.
inCoreAoS :: (PrimMonad m, RecVec rs) => Producer (Record rs) m () -> m (FrameRec rs) #
Stream a finite sequence of rows into an efficient in-memory
representation for further manipulation. Each column of the input
table will be stored optimally based on its type, making use of the
resulting generators a matter of indexing into a densely packed
representation. Returns a Frame
that provides a function to index
into the table.
inCoreAoS' :: (PrimMonad m, RecVec rs) => (Rec ((->) Int) rs -> Rec ((->) Int) ss) -> Producer (Record rs) m () -> m (FrameRec ss) #
toAoS :: Int -> Rec ((->) Int) rs -> FrameRec rs #
Convert a structure-of-arrays to an array-of-structures. This can simplify usage of an in-memory representation.
inCore :: forall m n rs. (PrimMonad m, RecVec rs, Monad n) => Producer (Record rs) m () -> m (Producer (Record rs) n ()) #
Stream a finite sequence of rows into an efficient in-memory representation for further manipulation. Each column of the input table will be stored optimally based on its type, making use of the resulting generator a matter of indexing into a densely packed representation.