Grid Computing

 

We literally invented Grid Computing.

We have the most sophisticated Grid offerings in the world.

 

In Science Tools' early years, we were fortunate to have significant business brought to us by word-of-mouth advertising. Some people had a good idea of what we do and how we can help them. But if you weren't one of those people, we often had a difficult time explaining to you what our primary product does. What we were lacking was a good catchy buzz-expression to capture people's imagination. Imagine our surprise when we began to investigate "Grid" and finally figured out that a key aspect of what we'd been doing all along was now known as this newfangled thing called "Grid Computing!" Yes, unbeknownst to him at the time, our Chief Scientist, Richard Troy, invented Grid Computing in 1995!

More than that, at the time we discovered this (late in 2004), we put some serious effort in to determining where "grid" was at the time, since we'd been hearing about it for some years already. What we discovered was that there existed only a vague goal and no functional systems that even begin to approach the promise of all that hype, except ours, of course. We have been delivering on the promise of grid since before the expression "grid computing" existed.

Now, what we do is more than just grid computing because, unlike (all?) other current practitioners of grid computing, our form includes built-in a high degree of contextual, semantic information generally known as "meta-data" - data about the data. We use this data to great effect, for example to help automate processing workflows, to help with user discovery of resources, and so forth.

There was another surprise for us, too. So far as we are aware, The BigSur System™ is the only turn-key grid computing solution in the world. Not only is it now, it always has been! And, our solution is heterogenous whereas most all the grid computing systems we've looked into are homogenous. The BigSur System™ includes the ability to identify what code belongs on what platforms, so you can deploy anywhere you have code that runs, and this can be a great advantage. For example, when older systems cannot be completely retired but must live side-by-side with younger systems, a single BigSur Server can manage the entire enterprise's computing needs. (BigSur has lots of other great computing advantages that can be found in the section on the Distributed Processing System.)

In fact, not counting the time it may take to install an RDBMS in the environment (like Postgres or Oracle), you can install The BigSur System™ in about 15 minutes, so if you have some program you want to run, you can install BigSur on a couple of systems, perform some configuring and be Grid Computing in something like an hour. In contrast, we have heard stories from people who say they have literally spent years trying to create a Grid Computing System using the latest tools and failed to reach their goals - and were forced to re-state their goals so they could save face in front of their management and peers.

Q: What do we mean when we say "turn-key?"

A: In our view, a system is "turn-key" when a user doesn't have to write any code - especially not any part of the system - to make use of it.

Q: Because it's intended to be a system that launches programs and runs them somewhere that's not local, clearly someone has to write that code - the application code. In Grid Computing, if the program is going to run just about anywhere, there will always be a need to fetch arguments and store results. Where's the boundary, then? Is fetching of data and returning of results the responsibility of the Grid System or the Application Program?

A: There's a shared responsibility for that, so one can argue that if the Application Code has to do some extra steps to fetch inputs and save results, that that programming is not counted and so such a system could be deemed "turn key" if that's all one had to do to use it.

Q: OK, so where does The BigSur System™ stand on this matter? How much code does one have to write?

A: The BigSur System™'s Java API has many sophisticated methods that make such things a complete no-brainer. Here's one example: getParameterAsFileWithIngest(int parameterIndex). This method is close to magic because it does so much for the caller. It knows the specified parameter applies to the currently running Process, so it knows that the index identifies the parameter exactly. It knows the Parameter Value was given before the Process was allowed to be run, so it knows that the Parameter is either a file or is a file object within The BigSur System™. Further, the meta-data tells BigSur where the file is, so if it's a file, it knows where to get it from. But it also knows that, due to the "WithIngest" part, the file is either already or is to be a BigSur Object - that's what ingestion does. So, there's a formal methodology by which that happens and as a part of that, it also establishes a file signature that uniquely identifies the file extraordinarily precisely. Then, if the file isn't already native to the local host computer, the method checks to see if the file happens to be in the local cache - and it doesn't confuse filenames, either(!), there's code to ensure it's right! - and if it's not available locally, it "gets" that file by transporting it from whereever it was to the local file system space.

The result returned to the caller of this method contains three key pieces of data: a boolean whether or not the method succeeded, a textual description of what happend as the method ran - not just for failures, but also including remote fetch data, etc, as appropriate, and it returns a string containing the argument's filename in the local file system space directly accessable to the Process.

One call. Does that sound like a lot of code to write? Saving results is similarly code-efficient.

 

PLEASE NOTE: If we have failed to answer all YOUR questions about Science Tools role in Grid Computing, please ask us directly.

Some references: