You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently the storage backend API isn't very formalized and it has some problems which would be nice to fix.
Firstly, the backend entrypoint it doesn't support ResourceContexts natively, instead it's just a single method which provides the user's callback with an open "dataset root". In Use ResourceContexts.jl for resource handling #12 I needed to invent ResourceContexts.enter_do to expose the data handle from within the do block but enter_do is kind of hacky/complex/inefficient as it needs to use a separate task stack.
Secondly, the "dataset root" type which is opened by the entrypoint has an informally defined API for use with our BlobTree abstraction; it's kind of complex and poorly distinguished from the internals of BlobTree itself, which makes it hard to implement and leads to suboptimal code reuse between backends.
Likely we should have an AbstractDataStorage type and perhaps define an open_storage function to go with this to solve the first problem. The second problem needs some joint refactoring of the existing tree storage backends to extract the common code.
The text was updated successfully, but these errors were encountered:
Currently the storage backend API isn't very formalized and it has some problems which would be nice to fix.
ResourceContexts
natively, instead it's just a single method which provides the user's callback with an open "dataset root". In Use ResourceContexts.jl for resource handling #12 I needed to inventResourceContexts.enter_do
to expose the data handle from within the do block butenter_do
is kind of hacky/complex/inefficient as it needs to use a separate task stack.BlobTree
abstraction; it's kind of complex and poorly distinguished from the internals ofBlobTree
itself, which makes it hard to implement and leads to suboptimal code reuse between backends.Likely we should have an
AbstractDataStorage
type and perhaps define anopen_storage
function to go with this to solve the first problem. The second problem needs some joint refactoring of the existing tree storage backends to extract the common code.The text was updated successfully, but these errors were encountered: