Methods

Compute Methods • Compute over Data • Local Compute • Web-Based Compute

Compute methods

We provide three types of computational methods to interact with data contained in a Node. Note that each of these methods are in prototype stage.

Send Compute to Data (Edge computing)

With Nodes, you will be able to send programs over to the data. This is known as edge computing and is especially powerful for data-intensive research. Nodes use Bacalhau, an architecture for fast, cost efficient, and secure computation to run jobs where the is generated and stored. A major advantage is that you don't need to rewrite your code - you can simply send it over to the data into an arbitrary docker container and WebAssembly (wasm) images as task.

Send Data to Compute

Import Node components such as (small) datasets and code into your local computer or high performance computing environment directly with Node PIDs. You can use the python library desci_fetch for this purpose. With it, you can import code and datasets from HTTP into your IDE. Please note that the library is under active development, feedback and contributors are welcome.

In-Browser Compute

Run code directly from your web browser in your Nodes #ide. Ideal for running results reproducibility checks or executing Jupyter notebooks.

Access Methods

Human Access over HTTP

To access content published on Nodes, refer to our PID schema. Nodes are resolved via the DNS resolver dpid.org.

Machine Access over HTTP

Open Beta Note

Please note that machine access over HTTP is currently under development.

We will soon be providing a syntax to enable machine resolution of the Node PID schema. This will allow consistent resolution to the machine-actionable manifest file of the Node.