Methods
Compute Methods • Compute over Data • Local Compute • Web-Based Compute
Last updated
Compute Methods • Compute over Data • Local Compute • Web-Based Compute
Last updated
We provide three types of computational methods to interact with data contained in a Node. Note that each of these methods are in prototype stage.
With Nodes, you will be able to send programs over to the data. This is known as edge computing and is especially powerful for data-intensive research. Nodes use Bacalhau, an architecture for fast, cost efficient, and secure computation to run jobs where the is generated and stored. A major advantage is that you don't need to rewrite your code - you can simply send it over to the data into an arbitrary docker container and WebAssembly (wasm) images as task.
Import Node components such as (small) datasets and code into your local computer or high performance computing environment directly with Node PIDs. You can use the python library desci_fetch for this purpose. With it, you can import code and datasets from HTTP into your IDE. Please note that the library is under active development, feedback and contributors are welcome.
Run code directly from your web browser in your Nodes . Ideal for running results reproducibility checks or executing Jupyter notebooks.
To access content published on Nodes, refer to our PID schema. Nodes are resolved via the DNS resolver dpid.org.
Open Beta Note
Please note that machine access over HTTP is currently under development.
We will soon be providing a syntax to enable machine resolution of the Node PID schema. This will allow consistent resolution to the machine-actionable manifest file of the Node.