lambdaloop

Scattered projects and ideas

Volunteer Computing Grid for Neuroscience

Doing neuroscience research requires a lot of computing power. Both simulating and analyzing neural data can be resource intensive. Research labs at universities (and increasingly, industry) have supercomputers for doing this. However, outside labs, it’s hard to get access to such resources and can often be the limiting factor for computational research.

Services like AWS EC2 have helped lower the barrier to entry for computing, but large computing projects can still be costly. For instance, a g2.2xlarge node (the minimum specs for deep learning) with 1 GPU, 8 cores, and 15GB of RAM, currently costs $0.65 / hour. Monthly, it costs $346 (with a discount). It can be useful for small experiments at a reasonable price ($6.5 for 10 hours), but longer experiments quickly become costly. Similar services have similar pricing.

So what’s the alternative?

A “grid” of computing power donated by volunteers, usable by any one, from anywhere.

Now, this sounds too good to be true and you probably already have some objections in your mind, so let me detail a possible architecture, with some potential problems and solutions. (All of the following is just speculation, as am I far from an expert on these systems. I’d love some feedback on anything, speak yourself in the comments below.)

Proposed Architecture

There are two types of people in the system, “users” and “volunteers”. Users submit programs to the system, whereas volunteers run them. In practice, it’s likely that a lot of the users will also be volunteers.

For users, submitting a script should be as simple as uploading a file to a webpage. It’s possible that these scripts may need external data, and they will definitely generate some results. We will need to setup an architecture to download data as needed for the analysis, and upload results when they are ready.

The actual system could be based on BOINC, which is the system behind projects like SETI@home and GPUGrid.net. It looks like BOINC already has some structure for making desktop grids for general purpose computing.

Another way to do this is to simply run code in the browser. The volunteers could go to a webpage which would execute some javascript code based on what the users submitted. The downside of this approach is that the users must submit some javascript code (or something that can be transpiled into javascript). The upside is that the barrier to entry for volunteers is dramatically lowered.

Some questions and answers

What about security?

So if the system is so flexible, can’t the users effectively upload any script to run on the volunteers’ machine? Isn’t that dangerous?

There are ways to sandbox programs, so that they do not use more than allocated resources. To what extent they would work in this scenario remains to be tested. If we are running the code in the browser, the browser itself would provide an effective sandbox.

Why would people volunteer their resources?

For science! No? Certainly this is enough to motivate a lot of people to volunteer their computing resources for the BOINC computing projects.

In our case, since the volunteers may also be users themselves, perhaps some kind of credit system can also be setup where depending on how much people donate resources, they are given higher/lower priority in the pool.